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EXECUTIVE  SUMMARY 

This  report  discusses  the  problems  of  integrating  Human  Factors  Engineering  (HFE)  into  the 
process  of  developing  Command,  Control,  and  Information  Systems  (CCIS).  User 
acceptance  of  CCIS  has  often  been  problematical,  and  reasons  quoted  for  such  problems 
include  lack  of  understanding  of  user  requirements  and  user  capabilities.  A  workshop  was 
organized  to  discuss  the  application  of  HOPE  in  the  development  of  command  and  control 
information  systems  for  the  Canadian  Forces  (CF).  The  workshop  concluded  that  a  user- 
centred  approach  should  be  taken  to  CCIS  development,  and  that  a  generic  Human 
Engineering  Programme  Plan  (HEPP)  should  be  developed  to  support  project  management 
of  HFE  in  CCIS. 

The  DND  Defence  Programme  Management  System  (DPMS)  includes  the  phases:  Project 
Planning,  Project  Development,  and  Implementation.  The  application  of  HFE  is  part  of  the 
Human  Systems  Integration  (HSI)  process  which  should  begin  in  the  Project  Planning 
stages.  The  DND  Project  Manager's  Handbook  covers  HFE  in  an  Annex  to  the  chapter  on 
Systems  Engineering  but  it  does  not  mention  HFE  in  the  chapter  on  software  development. 
DND  software  development  projects  are  usually  guided  by  US  DOD-STD-2167A  which 
does  not  address  HFE  directly,  although  various  HFE  regulatory  documents  are  cross- 
referenced  within  it. 

Opportunities  for  the  integration  of  HFE  in  the  software  engineering  process  are  provided  by 
new  system  design  methods.  METHOD/1™,  a  system  development  methodology 
published  by  Andersen  Consulting,  provides  a  detailed  decomposition  of  tasks  performed  in 
system  design  to  which  HSI  and  HFE  tasks  could  be  added  relatively  easily.  Object-oriented 
design  localizes  information  around  entities  referred  to  as  objects.  This  approach  allows  the 
human  to  be  considered  as  part  of  the  system,  with  the  possibility  of  including  human 
attributes  such  as  cognitive  work  style.  Existing  HFE  analysis  techniques  share  an  object- 
oriented  approach  with  this  method  of  software  engineering. 

User-centred  development  is  becoming  popular  as  an  approach  to  the  development  of 
information  systems,  and  several  variants  have  arisen.  The  common  elements  include  user 
involvement,  iterative  development  based  on  prototyping,  and  testing  against  usability 
criteria.  User  involvement  includes  identifying  requirements,  prioritizing  design  objectives, 
evaluating  prototypes,  and  establishing  validation  criteria.  Prototyping  is  a  key  feature  of 
the  user-centred  development  process.  A  prototype  provides  the  means  for  users  and  system 
designers  to  reach  a  common  understanding  of  system  requirements.  At  different  stages  of 
the  system  design,  prototypes  provide  the  opportunity  for  making  design  decisions  and 
evaluations.  Prototyping  is  not  without  its  risks.  When  poorly  planned  and  implemented, 
prototypes  will  fail  and  will  delay  a  project.  Prototyping  should  be  used  as  a  complement  to 
analysis  and  not  as  a  replacement  for  it. 

Within  any  project,  HFE  analysis,  experiment,  design,  and  test  and  evaluation  activities  are 
governed  by  the  project  Human  Engineering  Programme  Plan  (HEPP).  The  HEPP  defines 
the  requirements  for  applying  human  engineering  to  a  project,  including  integration  with 
system  engineering.  A  generic  HEPP  for  CCIS  can  be  based  on  the  US  specification  for  the 
application  of  HFE,  US  MIL-H-46855B.  Work  items  in  a  generic  HEPP  include: 
stakeholder  identification,  requirements  development,  analysis,  design,  interface 
specification,  prototyping,  iterative  development,  and  testing.  An  HEPP  also  details  the 
relationship  of  HFE  to  other  activities,  provides  a  list  of  deliverables,  and  includes  a  plan  for 
review  meetings.  A  generic  HEPP  will  invoke  various  regulatory  and  guidance  documents. 
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Details  of  modifications  which  should  be  made  to  MIL-H-46855B  for  use  in  CCIS 
development  are  provided  in  the  report. 

In  conclusion,  a  case  can  be  made  for  the  integration  of  HFE  in  DND  CCIS  projects. 
Difficulties  arise  in  integrating  HFE  with  system  engineering  because  the  documented 
approaches  to  software  development  do  not  address  HFE  sufficiently.  New  approaches  to 
software  engineering  and  a  user-centred  approach  provide  a  promising  combination  for 
improving  the  situation.  The  key  to  a  user-centred  approach  is  prototyping,  and  the  effective 
implementation  of  a  user-centred  approach  depends  on  a  HEPP.  The  specification  US  MIL- 
H-46855B  provides  a  good  framework  for  developing  a  generic  HEPP. 
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ABSTRACT 

The  importance  of  integrating  Human  Factors  Engineering  with  the  development  of 
Command,  Control  and  Information  Systems  (CCIS)  is  discussed.  It  is  recommended  that  a 
user-centred  approach  be  taken  to  the  development  of  such  systems.  The  essential  features 
of  four  software  development  approaches  are  reviewed.  It  is  concluded  that  current 
approaches  do  not  facilitate  taking  a  user-centred  approach,  but  new  software  system  design 
methods  provide  may  do  so.  The  essential  features  of  a  user-centred  approach  are  outlined, 
and  the  contribution  of  prototyping  is  reviewed.  Modifications  to  an  existing  Human 
Engineering  Programme  Plan,  US  MIL-STD-46855B,  are  suggested,  to  provide  the  basis  for 
a  plan  for  a  user-centred  approach  in  CCIS. 
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INTRODUCTION 


The  Department  of  National  Defence  (DND)  is  involved  in  the  development  of  a  number  of 
computer-based  command  and  control  systems,  known  as  Command,  Control  and 
Information  Systems  (CCIS).  It  is  estimated  that  up  to  45%  of  software  development  costs 
are  associated  with  the  user  interface  (Nielsen,  1993).  Yet,  there  have  been  many  problems 
with  user  acceptance  and  the  ease  of  use  of  CCIS  (see  Hendricks  et.  ah,  1983,  for  example) 
and  commercial  Information  Technology  (IT)  systems.  These  problems  can  be  addressed 
through  the  application  of  human  factors  engineering  (HFE) l,  which  contributes  to  human 
systems  integration  (HSI)  (US  DoD,  1991).  This  report  addresses  the  application  of  HFE  to 
CCIS  acquisition,  in  a  user-centred  approach,  to  avoid  such  problems. 

A  number  of  reasons  have  been  cited  for  problems  in  the  development  of  CCIS  and  IT 
systems,  including  lack  of  understanding  of  users’  requirements  and  capabilities  (see  Rouse, 
1991,  for  example).  Understanding  users’  requirements  is  difficult,  because  the  behaviour  of 
any  new  system  is  emergent,  depending  on  the  interaction  between  what  technology  dictates 
that  the  users  do  and  what  the  users  do  to  exploit  the  technology.  The  introduction  of  new 
technology  changes  the  users’  understanding  of  what  they  can  do  and  what  they  need.  So 
user  requirements  for  complex  man-machine  systems  are  often  subjective,  vague, 
incomplete,  or  unknown  (McLaughlin,  1987;  Pressman,  1987). 

Understanding  users’  capabilities  and  limitations  can  also  be  difficult.  CCISs  are 
comparatively  new,  and  designers  do  not  have  the  general  understanding  of  the  associated 
human  functions  and  tasks  that  the  designers  of  more  established  systems  such  as  aircraft 
and  armoured  vehicles  have.  Compared  with  other  military  systems,  CCISs  involve  many 
users  having  a  wide  range  of  skills  and  experience.  The  experts  in  computer  science  and 
engineering  who  develop  CCISs  may  not  appreciate  the  differences  between  themselves  and 
real  system  users.  This  can  range  from  ‘atomic  level’  differences  in  formatting  data  such  as 
dates  and  times  to  user  capabilities  such  as  typing  and  map  reading  and  job-related  issues 
such  as  the  responsibility  for  authorizing  messages. 

In  addition,  although  guidelines  for  the  details  of  the  human-computer  interface  have  been 
developed  (Defense  Information  Systems  Agency,  1992,  for  example)  exhaustive,  generic, 
guidelines  for  the  design  of  CCISs  cannot  be  developed  independently  from  the  operators’ 
tasks  (Behavioural  Team,  1991;  NATO  AC/243  (Panel-8),  1989).  Thus,  software 
development  may  evolve  beyond  the  concept  definition  phase  before  user  requirements  are 
sufficiently  understood  to  develop  an  effective  user-computer  interface  (Overmyer,  1990). 

The  difficulties  outlined  above  hinder  the  specification  of  software  of  the  complexity 
required  for  CCISs.  As  a  result,  risk  is  incurred  in  the  selection  of  existing  software 
packages,  or  Commercial  Off-the-Shelf  (COTS)  systems  for  the  Canadian  Forces  (CF)  and 
the  development  of  new  systems  is  hindered. 

These  issues,  and  the  need  to  provide  advice  on  the  management  of  the  human  engineering 
process  were  discussed  during  a  workshop  on  ‘The  application  of  human  factors  engineering 
in  the  development  of  command  and  control  information  systems  for  the  Canadian  Forces’ 
(Beevis,  Essens  &  Mack,  1993).  One  conclusion  of  the  workshop  was  that  a  generic  human 
engineering  programme  plan  (HEPP)  is  required  which  provides  the  structure  to  support 


1  also  called  human  engineering  -  HE 
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PMO  personnel  in  the  specification  of  HFE  and  the  management  of  CCIS  development  and 
acquisition. 

This  report  addresses  the  integration  of  HFE  with  the  development  of  CCISs.  The  user- 
centred  development  process  is  presented.  A  draft  of  a  generic  project  plan  for  the 
application  of  HFE  in  CCIS  is  provided. 

THE  BUSINESS  CASE  FOR  HFE  IN  CCIS 

From  the  foregoing  material  it  can  be  seen  that  HFE  might  make  a  major  contribution  to 
ensuring  the  operability  and  usability  of  new  systems  2.  Although  relevant  data  are  scarce, 
the  application  of  human  factors  and  HSI  can  result  either  in  improvements  to  system 
performance  (or  system  effectiveness)  or  in  cost  savings  (Beevis  &  Slade,  1970).  Thus  a 
business  case  can  be  developed  for  taking  a  user-centred  approach  to  systems  acquisition. 
The  business  case  for  the  application  of  human  factors  in  IT  systems  includes  its  effects  on 
sales  (Chapanis,  1991a),  which  is  not  an  issue  in  CCIS.  Business  cases  are  usually  based  on 
three  categories: 

•  Costs  saved 

•  Costs  avoided 

•  New  opportunities. 

For  IT  systems,  ‘costs  saved’  are  often  realized  through  reduced  personnel  costs.  A  typical 
human  resources  investment  model  for  military  systems  (Booher  &  Rouse,  1990)  shows 
several  areas  where  human  resources  costs  may  be  saved.  These  include:  overall  numbers 
required;  rejection  rates  through  selection  and  training;  and  job  performance.  The  aim  of  HSI 
is  to  minimize  the  life-cycle  costs  of  a  system  through  control  and  tradeoffs  between 
numbers  of  personnel,  skills  required,  training  required,  system  safety  considerations  and 
human  factors  engineering.  The  goal  of  HFE  is  to  match  a  system  to  the  capabilities  and 
limitations  of  the  anticipated  operators,  and  maintainers,  and  to  ensure  that  system 
personnel  do  function  effectively. 

‘Cost  avoidance’  can  be  achieved  by  not  putting  into  the  system  features  that  will  not  be 
used,  or  by  avoiding  cost  overruns.  Many  IT  system  cost  overruns  are  associated  with 
requests  for  changes  by  users,  overlooked  operator  tasks,  user’s  lack  of  understanding  of 
their  own  requirements,  and  insufficient  user-developer  communication  and  understanding 
(Nielsen,  1993).  Cost  avoidance  is  also  equivalent  to  a  Quality  Assurance  approach: 
problems  in  operating  the  system,  and  the  need  for  their  rectification,  are  avoided.  For 
example,  the  thrust  of  Artillery  Regimental  Data  System/Advanced  Development  Model  is 
to  avoid  problems  with  the  current  manual  system,  particularly  the  ‘costs’  of  delays  and 
errors.  As  another  example,  many  new  military  systems  have  resulted  in  ‘skill  creep’  where 
successive  systems  have  required  higher  levels  of  skill,  experience  and/or  training  than 
predecessor  systems.  The  application  of  an  integrated  HSI  and  HFE  effort  can  contribute  to 
avoiding  such  costs  in  new  systems. 

For  a  military  application,  'new  opportunities'  can  be  realized  by  insuring  that  the  system  is 
more  flexible.  This  requires  attention  to  the  human  roles,  functions  and  tasks  to  ensure  that 
they  can  behave  in  a  flexible  way,  rather  than  being  constrained  by  the  system  design.  In 
some  cases,  system  availability  may  be  increased.  For  example,  day/night,  all-weather 


2  Usability  has  been  defined  as  the  set  of  attributes  that  bear  on  the  effort  required  to  use  (learn,  understand 
and  operate)  software  and  on  the  individual  assessment  of  such  use  by  a  stated  or  implied  set  of  users  (APEO, 
1993). 
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operation  CCIS  operation  may  be  facilitated,  or  downtime,  diagnosis  and  repair  times 
reduced,  or  it  may  be  possible  for  to  operate  for  longer  periods  without  user  fatigue  affecting 
the  system. 

The  gains  in  savings,  avoided  costs  and  system  flexibility  can  be  evaluated  against  the  costs 
of  the  HFE  effort  Direct  HFE  costs  for  US  army  systems  were  estimated  at  between  3  and 
5%  of  project  design  costs  in  the  1960s;  recently  the  costs  for  HSI  activities,  which  are  wider 
in  scope  than  HFE,  were  estimated  at  a  similar  level,  with  the  highest  value  of  8%  (Booher 
&  Rouse,  1990).  In  the  development  of  commercial  IT  systems,  a  median  value  of  6%  of 
budget  relative  to  total  has  been  found  in  a  survey  of  3 1  development  projects  (Nielsen, 
1993).  The  same  source  reports  increases  in  productivity  of  12%,  and  pay-backs  of  the 
investment  in  HFE  ranging  from  536%  in  one  year  to  200%  in  the  first  day  of  operation. 


THE  SYSTEM  DEVELOPMENT  PROCESS 

DND  acquisition  projects  are  governed  by  the  Defence  Programme  Management  System 
(DPMS),  as  outlined  in  Figure  1.  Given  the  importance  of  establishing  the  user’s 
requirements  for  CCIS,  HSI  analysis  should  start  in  the  Project  Planning  stage  of  a  project. 
Requirements  for  system  operability,  availability  and  maintainability  defined  at  that  stage 
will  determine  HSI  requirements  such  as  manning  levels,  skill  levels  and  training 
requirements. 

The  System  Specification  (SS)  must  include  HSI  and  HFE  requirements  in  more  detail, 
especially  for  off-the-shelf  acquisitions.  (MIL-STD-1521B  ‘Technical  reviews  and  audits  for 
systems,  equipments,  and  computer  software’  does  not  address  the  need  to  include  HSI  and 
HFE  requirements  into  the  SS,  although  is  does  include  ‘Human  Factors  Analysis,’  ‘Life 
Cycle  Cost  Analysis,’  and  ‘Manpower  Requirements/Personnel  Analysis’  in  the  System 
Requirements  Review  (SRR)). 

A  major  part  of  the  development  of  CCIS  involves  software.  Typically,  software  is 
developed  during  the  Implementation  phase  of  a  project  (CFP  A-LP-005-000/AG-006, 

DND,  1990). 

Software  Engineering  And  Human  Factors  Engineering 

While  software  engineers  recognize  the  importance  of  HFE,  the  time  and  budget  constraints 
of  a  development  effort  often  force  HFE  issues  to  a  lower  priority  in  the  software 
engineering  process.  This  observation  is  particularly  true  for  project  teams  that  lack  HFE 
expertise  or  experience.  Much  of  the  documentation  governing  the  development  of  systems 
and/or  software  does  not  emphasize  HSI  or  HFE.  For  example,  MIL-STD-881A  ‘Work 
breakdown  structures  for  defense  military  items’  does  not  identify  HFE  or  HSI  effort  as  a 
work  item;  instead  ‘human  engineering’  is  treated  as  a  ‘speciality  engineering’  activity 
within  systems  engineering.  Recent  observation  of  the  ARDS/ADM  project  showed  that 
even  with  PMO  encouragement,  the  integration  of  HFE  with  other  systems  engineering 
activities  is  far  from  assured. 

HFE  specialists  can  help  raise  the  priority  of  HFE  by  suggesting  methods  of  including  HFE 
techniques  into  existing  software  engineering  structures.  This  section  explores  how  HFE 
might  be  integrated  into  several  relevant  software  engineering  methodologies.  The 
techniques  discussed  below  are  all  in  use  in  Canada  or  other  NATO  countries.  The  ideas 
presented  here  are  examples  only;  significant  work,  beyond  the  scope  of  this  report,  would 
be  required  to  fully  integrate  HFE  with  the  software  engineering  effort. 
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Figure  1 :  Defence  Programme  Management  System  phases  and 
associated  management  tasks 


1.  DND  Approach  to  software  development 

The  overall  approach  to  system  development  is  guided  by  the  DND  Project  Manager’s 
Handbook  (A-LP-005-000/AG-001  to  007).  Sections  of  that  handbook  deal  with  project 
management,  systems  engineering,  and  logistics  support.  Chapter  35  deals  with  Weapon 
System  Software  Management.  The  material  relates  the  Canadian  DPMS  and  the  Life 
Cycle  Management  System  phases  to  US  DOD-STD-2167A  on  software  development. 
Advice  is  given  on  the  avoidance  of  problems  with  software  development.  The  management 
of  software  development  during  Project  Development,  Project  Definition  and  Project 
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Implementation,  and  the  In-Service  Stage  is  reviewed,  and  work  items  and  products 
described.  Six  basic  software  development  activities  are  identified. 

Chapter  35  does  not  mention  HFE  as  a  software  development  activity  (HFE  is  dealt  with  as 
an  Annex  to  Chapter  30  -  System  Engineering).  Six  typical  problems  that  occur  with 
software  development  are  listed,  the  first  of  which  is  “original  requirements  that  are 
incomplete  and/or  invalidated.”  The  material  classifies  software  errors  into  three  categories: 

•  requirement  errors 

•  design  errors, 

•  coding  errors. 

It  is  recommended  that  the  “RFP  should  be  given  special  attention  to  ensure  that  the 
‘essential  ingredients’  are  defined  to  ensure  visibility  and  control  by  the  material  developed 
for  the  development  effort.”  It  is  also  recommended  that  the  contractor  submit  plans  for 
matters  such  as  software  development,  configuration  management,  design  review,  quality 
assurance,  and  any  simulation  and  test  facilities  with  the  original  proposal,  rather  than  some 
months  into  the  contract.  This  parallels  DND  experience  with  the  documentation  of  the 
human  factors  programme  plan  (HEPP).  Experience  shows  that  it  is  preferable  to  have  the 
HEPP  as  part  of  the  proposal,  rather  than  have  it  negotiated  several  months  after  contract 
award  when  commitments  have  been  made  to  other  contract  work  items. 

2.  US  DOD-STD  2I67A  Defense  Systems  Software  Development 

This  standard  “establishes  uniform  requirements  for  software  development  that  are 
applicable  throughout  the  system  life  cycle  [the  acquisition,  development,  or  support  of 
software  systems] ....  [which]  provide  the  basis  for  Government  insight  into  a  contractor’s 
software  development,  testing,  and  evaluation  efforts.  ...  It  is  not  intended  to  specify  or  to 
discourage  the  use  of  any  particular  software  development  method.  The  standard,  should  be 
used  in  conjunction  with  MIL-STD-499,  Engineering  Management.”  DOD-STD-2167A 
identifies  the  project  phases,  deliverables  and  criteria  for  evaluating  them,  progress  reviews, 
and  records  which  must  be  kept  by  the  contractor.  The  project  phases  (Figure  2)  can  be 
related  the  to  six  basic  software  development  cycle  activities  identified  in  the  Chapter  35  of 
the  DND  Project  Manager’s  Handbook. 

DOD-STD-2167A  is  based  on  a  phased  development  process: 

a.  System  Requirements  Analysis/Design 

b.  Software  Requirements  Analysis 

c.  Preliminary  Design 

d.  Detailed  Design 

e.  Coding  and  CSU  Testing 

f.  CSC  Integration  and  Testing 

g.  CSCI  Testing 

h.  System  Integration  and  Testing 

As  shown  in  Figure  2,  the  standard  relates  the  various  design  reviews  and  the  deliverables 
associated  with  those  activities  to  the  development  phases.  It  does  not  cover  the  initial  stages 
of  the  DPMS,  or  later  stages  of  life-cycle  management  (ADGA,  1994). 

Deliverables  are  specified  by  18  Data  Item  Descriptions  (DIDs)  covering  plans,  design 
documentation,  tests,  and  manuals  (Appendix  A).  The  standard  also  identifies  the  types  of 
criteria  for  evaluation  of  deliverables.  The  requirements  for  documentation  of  the  software 
development  in  terms  of  its  decomposition,  traceability  to  design  requirements,  design 
standards  used,  performance  requirements,  test  requirements,  configuration  management, 
etc.  are  identified. 
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Figure  2:  System  development  reviews  and  audits  suggested  by  DOD-STD-21 67 A 


Standard  2167A  is  cross-referenced  to: 


DOD-STD-480 

MIL-STD-48 1 

MIL-STD-490 

MIL-STD-499 

MEL-STD-1521 


Configuration  Control  -  Engineering  Changes,  Deviations, 
and  waivers 

Configuration  Control  -  Engineering  Changes,  Deviations, 
and  waivers  (short  form) 

Specification  Practices 
Engineering  Management 

Technical  Reviews  and  Audits  for  Systems,  Equipments, 
and  Computer  Software 


The  standard  does  not  address  HFE  directly.  However,  MIL-STD-490,  MIL-STD-499  and 
MIL-STD-1521B  do  address  HFE,  and  an  HFE  management  plan  can  be  produced  which  is 
compatible  with  DOD-STD-21 67A,  as  follows: 


•  MIL-STD-490  includes  standard  paragraphs  for  the  inclusion  of  Human 
Engineering  and  Personnel  and  Training  requirements  in  system  specifications 

•  MIL-STD-499  requires  that  Engineering  Speciality  Integration  ensure  “the  timely 
and  appropriate  intermeshing  of  engineering  efforts  and  disciplines  such  as 
reliability,  maintainability,  logistics  engineering,  human  factors,  safety,  value 
engineering,  standardization,  transportability,  etc.  to  insure  their  influence  on 
system  design.” 

•  MIL-STD-1521B  covers  seven  project  reviews  and  three  audits.  The  project  reviews 
cover  Human  Factors  Analysis  as  a  topic,  from  the  System  Requirements  Review 
(SRR)  onwards.  The  review  also  covers  other  aspects  of  HSI,  including  logistics 
support  analysis,  and  Manpower  Requirements/Personnel  Analysis. 
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An  iterative  approach  to  system  development  could  be  accommodated  within  the  2 167 A 
framework  provided  that  iteration  is  limited  to  the  specific  project  stages.  Metersky  (1993) 
proposes  a  modification  to  the  2167A  pattern  of  development  which  expands  the  Software 
Requirements  Analysis  and  Preliminary  Design  phases  to  include  the  following  steps: 

•  Specify  mission  requirements  or  operational  problems 

•  Identify  operational  user  community 

•  Perform  operational  user  requirements  analysis 

•  Perform  functional  requirements  analysis 

•  Develop  preliminary  system  architecture 

•  Build  a  prototype 

•  Test  in  lab 

•  Test  in  operational  environment:  solicit  refinements  and  extensions 

•  Implement  revisions/enhancements 

•  Reiterate  incremental  developments 


3.  METHOD/1™ 

METHOD/1  is  the  registered  trademark  of  a  toolset  produced  by  Andersen  Consulting  for 
developing  software  systems.  The  tools  have  been  selected  by  DND  as  part  of  a  proposed 
standard  System  Development  Methodology  (SDM): 

“This  directive  identifies  the  use  of  the  METHOD/1,  the  Systems  Development 
Methodology  as  mandatory  for  any  project  developing  or  acquiring  a  multi-user 
information  system  that  includes  an  application  software  component  and  that  is 
not  exclusively  a  process  control  system.  “(DIS  Arch,  1993) 

METHOD/1  is  supported  by  CASE  software.  The  software  includes  much  of  the 
documentation,  estimating  tools,  and  project  management  tools.  DND  is  supporting  the 
distribution  and  use  of  the  software  and  techniques. 

METHOD/1  is  a  functional  decomposition  of  the  software  life  cycle.  At  the  top  level  are 
phases,  including:  Information  Planning,  Custom  Systems,  Packaged  Systems  (i.e., 
Commercial  Off-the-Shelf  or  COTS),  Iterative  Systems,  and  Support.  Some  phases  are 
optional  and  some  are  exclusive  of  others.  For  example,  a  system  is  (typically)  primarily 
either  a  Custom  system  or  a  Packaged  system.  Each  phase  is  decomposed  into  segments  and 
segments  are  further  broken  down  into  tasks;  within  a  task  can  be  found  work  steps. 

Two  aspects  of  MEIHOD/1  make  it  attractive  for  the  software  life  cycle.  First,  there  is 
extensive  support  for  management  of  software  projects.  Systems  Development  Management 
activities  occur  in  parallel  with  all  phases  of  METHOD/1.  Project  managers  are  provided 
with  guidance  constructing  business  cases,  selecting  project  teams,  estimating  system  costs 
and  time  lines,  etc.  A  project  manager  need  not  be  a  software  engineer  to  work  effectively 
with  METHOD/1.  Second,  METHOD/1  provides  extensive  examples  of  management 
reports,  deliverables,  and  checklists  called  'object  samples'.  While  the  samples  are  not 
intended  to  be  a  comprehensive  set  of  system  definition,  description,  and  documentation  - 
they  have  in  practice  been  directly  invoked  by  project  managers. 

Human  Engineering  activities  could  be  integrated  into  METHOD/1  using  the  functional 
decomposition  approach.  For  example,  Beevis  et  al.  (1993,  Figure  93)  present  a  list  of  work 
items  in  the  human  factors  engineering  process  which  parallel  METHOD/1  segments.  The 
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same  report  also  contains  a  list  of  deliverables  (ibid.,  Figure  94)  which  could  be  the  basis  for 
METHOD/1  object  samples  and  linked  to  METHOD/1  segments.  While  significant  work 
would  likely  be  involved,  the  result  would  offer  project  managers  a  well  defined  and  easily 
managed  way  to  integrate  human  engineering  techniques  into  the  software  life  cycle. 


4.  Hatley-Pirbhai 

Hatley-Pirbhai  (H-P)  is  a  published  method  for  real  time  system  specification  (Hatley  and 
Pirbhai,  1988).  The  method  was  developed  to  address  a  deficiency  in  structured  analysis 
techniques  that  was  identified  during  development  of  a  large  avionics  system.  The  Hatley- 
Pirbhai  method  was  selected  by  the  ARDS/ADM  contractor  as  the  structured  analysis  and 
design  method  for  the  project.  The  H-P  method  covers  only  the  early  stages  of  the  software 
engineering  process;  it  does  not  include  system  implementation  and  testing.  Therefore,  a 
complete  integration  of  HFE  methods  cannot  be  built  around  H-P.  However,  with  that 
reservation,  the  H-P  system  is  discussed  below. 

Hatley-Pirbhai  recognizes  that  system  development  is  an  iterative  process,  an  important 
concept  for  user-centred  design  and  the  successful  integration  of  HFE.  An  overlapping  cycle 
of  requirements,  design,  implementation  and  test  is  noted  in  the  reference.  Requirements  are 
addressed  by  the  construction  of  Data  Flow  Diagrams  (DFDs)  and  Control  Flow  Diagrams 
(CFDs).  The  requirements  are  then  transformed  into  an  architecture  model  by  considering 
design  constraints  such  as  performance,  expansion  capability,  safety  and  reliability. 

Consideration  of  the  human  in  the  system  is  included  in  requirements  model 
construction.  Users  are  categorized  as  owners,  operators,  maintainers,  designers  and 
builders.  The  role  of  each  is  described  in  general  terms.  Users  are  also  identified  as  the 
major  source  of  system  requirements,  in  the  form  of  statements  of  needs.  Some  of  the 
difficulties  involved  in  collecting  and  defining  these  statements  are  discussed  briefly,  but 
little  guidance  is  provided. 

It  is  difficult  to  imagine  a  complete  integration  of  HFE  into  Hatley-Pirbhai.  In  book  form, 
H-P  provides  great  detail  concerning  the  construction  of  requirements  and  architecture 
models  using  structured  techniques.  Consideration  of  HFE  at  various  stages  of  model 
construction  would  have  to  be  treated  separately,  with  references  to  the  H-P  process. 
Interpretation  and  perhaps  modification  of  the  H-P  stages  would  be  required.  The  result 
would  likely  be  a  'take-it  or  leave-it'  compendium  with  weak  links.  Successful  use  of  HFE 
with  the  H-P  system  would  likely  require  an  HFE  expert  on  the  design  team. 


5.  Object-Oriented  Software  Engineering 

Object-Oriented  software  engineering  is  quickly  growing  in  use  and  popularity.  DND 
contractors  are  studying  and  using  the  techniques.  The  ARDS/ADM  contractor  identified 
the  need  to  relate  Object  Oriented  Programming  (OOP)  and  HFE  (Beevis  et  al.,  1993).  The 
United  States  Air  Force's  Armstrong  Laboratory  has  issued  a  standard  for  Object-Oriented 
system  design,  recognizing  the  growing  popularity  of  the  method  (Mayer,  Keen  &  Wells, 
1992).  OOP  can  be  characterized  by  three  attributes  (Berard,  1993): 

•  Information  is  localized  around  objects,  where  an  object  is  a  thing  or  a  pattern  of 
things 

•  Objects  are  complete  entities,  containing  as  complete  a  description  as  possible;  and 

•  Objects  are  independent  of  each  other,  but  can  be  arranged  in  a  hierarchical  manner. 
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This  broad  definition  of  object  includes  things  like  Organization,  Operator,  and  Event.  A 
group  of  objects  with  common  attributes  is  defined  as  an  object  class.  Depending  on  the 
model,  any  one  of  the  above  objects  could  be  a  class.  For  example,  Organization  might  be  a 
class  containing  Service-Organization,  Manufacturing-Organization,  and  Government- 
Organization.  Members  of  an  object  class  inherit  attributes  of  the  class-object.  The 
Organization  class  might  have  attributes  such  as  number-of-members,  name,  and  annual- 
budget.  Each  of  the  more  specific  types  of  organization  inherits  these  attributes. 

One  Object-Oriented  development  system  is  Smalltalk/V™  by  Digitalk™.  The  introduction 
to  the  Smalltalk/V  Mac  documentation  contains  the  following  (Digitalk,  1991): 

"Smalltalk  grew  from  a  few  powerful  ideas. 

•  The  most  important  component  in  a  computing  system  is  the  individual  human  user. 

•  Programming  should  be  a  natural  extension  of  thinking. 

•  Programming  should  be  a  dynamic,  evolutionary  process  consistent  with  the  model 
of  human  learning  activity." 

This  quote  could  easily  have  been  taken  from  a  text  on  human-computer  interaction.  OOP 
concepts  align  closely  with  those  of  human  engineering. 

The  concepts  of  OOP  were  developed  in  the  1960s  and  1970s.  More  recently  object- 
oriented  approaches  to  the  analysis  and  design  stages  of  the  software  life  cycle  have  emerged 
(Coad  and  Yourdon,  1990).  Object-Oriented  Analysis  (OOA  -  also  called  Object-Oriented 
Requirements  Analysis  -  or  OORA)  and  Object-Oriented  Design  (OOD)  have  evolved  from 
the  need  to  blend  changing  programming  preferences  (i.e.,  OOP)  with  the  traditional 
software  life-cycle. 

The  object-oriented  approach  to  software  systems  holds  promise  for  the  integration  of 
Human  Engineering  concepts.  OOA  includes  the  identification  of  major  objects  in  a  system 
-  an  excellent  opportunity  to  include  the  human  in  the  system  concept.  Characteristics  of 
major  objects  are  identified  and  included  in  object  definition.  Some  basic  examples  of 
human  attributes  are  name,  age,  and  address.  However,  human  attributes  could  include,  for 
example,  cognitive  work  style,  personality  type  and  'handed-ness'  -  items  less  frequently 
associated  with  software  design. 

A  Human  Engineering  analysis  of  a  system  will  often  include  a  task  analysis.  One  method. 
Task  Analysis  for  Knowledge  Descriptions  (TAKD,  Johnson,  Diaper  &  Long,  1984)  refers 
to  'specific  objects',  which  are  defined  as:"...  all  objects  that  are  relevant  to  the  performance 
of  a  task,  or  set  of  tasks.  The  definition  of  an  object  is  somewhat  problematical  in  that  a 
specific  object  may  be  a  part  of  a  larger  object"  (Diaper,  1989).  This  example  shows  that 
object-oriented  techniques  allow  for  system  definition  in  similar  terms  as  those  used  in  task 
analysis.  A  task  analysis  system  called  'Analysis  for  Task  Object  Modeling'  (ATOM)  has 
been  developed  to  aid  users  of  traditional  structured  design  methods  in  considering  user 
interface  design  (Walsh,  1989).  ATOM  descriptions  are  created  by  "...  analyzing  principal 
objects  of  the  system,  and ...  identifying  the  major  actions  associated  with  each  object." 

These  activities  strongly  parallel  object-oriented  concepts. 

One  of  the  three  attributes  of  OOP  noted  above  is  the  localization  of  information  around 
objects.  Included  in  this  concept  are  methods,  or  actions  which  can  affect  an  object.  The 
Human  Engineering  process  of  functional  decomposition  usually  results  in  a  list  of  actions 
by  a  particular  operator.  In  OOA,  each  operator  may  be  represented  by  an  object  and  tasks 
assigned  to  that  operator  may  be  represented  by  that  object's  methods. 
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In  summary,  an  object-oriented  approach  to  software  systems  is  highly  compatible  with 
human  engineering  activities.  The  human  can  be  viewed  as  part  of  the  system,  integrating 
human  engineering  with  system  engineering.  Human  engineering  information  such  as 
cognitive  work  style  and  tasks  performed  by  each  operator  can  be  encapsulated  in  an  object- 
oriented  model. 

Conclusions 

Overall,  the  material  reviewed  does  not  place  a  strong  emphasis  on  HFE  or  user 
requirements  or  their  integration  with  software  development  activities.  The  review  shows, 
however,  that  HFE  can  be  included  into  various  approaches  to  software  development 
provided  that  such  activities  are  planned  for. 


THE  USER-CENTRED  DEVELOPMENT  PROCESS 

The  experts  who  attended  the  workshop  on  ‘The  application  of  human  factors  engineering  in 
the  development  of  command  and  control  information  systems  for  the  Canadian  Forces’ 
(Beevis  et  al.,  1993)  agreed  that  adopting  a  user-centred  approach  to  information  system 
development  provides  the  best  way  to  ensure  proper  consideration  of  human  factors  issues. 

The  ‘user-centred’  approach  has  not  been  clearly  defined  in  the  way  that  ‘human  factors’  and 
‘operability’  have  (DND-ENG.  STD-3, 1969).  A  user-centred  approach  can  be  considered  as 
focussed  on  human  system  integration.  Such  an  approach  would  provide  the  answers  to  the 
questions  posed  by  the  US  Army’s  HSI  (MANPRINT)  programme  : 

“Can  this  soldier 

with  this  training 

perform  these  tasks 

to  these  standards 

under  these  conditions?” 

US  Department  of  the  Army,  1990). 

Approaches  to  user-centred  development  have  been  called  ‘multi-disciplinary  information 
systems  engineering’  (Andriole,  1990),  ‘user  engineering  methodology’  (McLaughlin, 

1987),  ‘user-centred  system  design’  Norman  &  Draper,  1986),  and  ‘usability  engineering’ 
(Whiteside,  Bennett  &  Holtzblatt,  1988;  Nielsen,  1993). 

Andriole's  (1990)  approach  supports  the  principles  of  top-down  design,  and  hierarchical 
decomposition,  synthesis,  iteration  and  assessment.  It  calls  for  extensive  use  of  prototyping 
and  stresses  the  importance  of  identifying  user  requirements  before  software  requirements 
are  specified. 

McLaughlin's  (1987)  approach  emphasizes  iterative  development  through  user  evaluation  of 
a  prototype.  His  ‘user  engineering  methodology’  has  been  combined  with  traditional  system 
engineering  techniques  to  develop  complex  man-machine  systems.  The  approach  is  intended 
to  gather  data  about  the  potential  system  users  and  incorporate  those  data  into  the  design 
process  as  early  as  possible.  The  methodology  emphasizes,  defines,  validates,  and  maintains 
the  user’s  view  of  the  system  being  developed. 

Thus  user-centred  approaches  emphasize  problem  definition  and  requirements  development 
as  well  as  iterative  development  and  testing  (Williges,  Williges  &  Elkerton,  1987).  An 
outline  of  the  major  activities  in  such  a  programme  is  shown  in  Table  1. 
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Key  features  of  these  approaches  include: 

•  a  ‘user  analysis’  to  derive  a  model  of  the  user  group 

•  the  development  of  specifications  which  include  ‘usability’  requirements 

•  evolution  of  the  design  through  iteration 

•  testing  the  design  through  prototyping  using  ‘usability’  criteria 

The  current  report  is  written  to  advocate  an  iterative  approach  to  the  user-centred 
development  process  and  to  summarize  relevant  information  draw  from  several  sources.  The 
information  on  a  user-centred  approach  includes: 

•  user  involvement  and  establishing  user  profiles 

•  prototyping  and  iterative  development  in  concert  with  analysis 

•  testing  against  usability  criteria 


Table  1 :  Work  items  for  the  user-centred  approach  to  systems 

1.  ITERATIVE  PROBLEM  DEFINITION 

•  identification  of  stakeholders  and  their  relationships 

•  identification  of  problems  &  goals 

•  exploration  of  goals  -  what  can  be  achieved  with  certainty 

-  what  it  is  hoped  can  be  achieved 

-  what  it  is  desirable  to  achieve 

•  identification  of  organization  implications 

•  development  of  a  user  participation  plan 

2.  ITERATIVE  REQUIREMENTS  DEVELOPMENT 

•  user  analysis 

•  system,  organizational  &  training  analysis 

•  specification  of  performance  goals  &  criteria 

•  concept  development,  allocation  of  functions,  concept  exploration  &  demonstration 

3.  ITERATIVE  DESIGN/PROTOTYPE  &  TEST 

•  detailed  task  analysis 

•  detailed  performance  goals 

•  system,  interface  &  training  system  demonstration  &  design 

•  evaluation  of  a  functional  prototype  (at  SDR,  PDR  &  CDR) 

4.  TRANSITION  OF  PROTOTYPE  TO  PRODUCTION 

5.  FIELD  TFtlAL 


User  Involvement  and  Establishing  User  Profiles 

The  specification  for  HFE  activities,  US  MBL-H-46855B  para  3.2.3.2  of  Test  and  Evaluation 
requires  that  "Use  of  military  personnel  from  the  intended  user  population  is  preferred  where 
feasible."  However,  user-centred  iterative  development  involves  more  than  just  employing 
users  as  subjects  in  acceptance  tests.  A  range  of  user  representatives  is  required  throughout 
the  project  life-cycle.  This  includes  the  user  groups  which  are  sponsoring  the  development, 
the  end  user  agencies,  the  users'  representatives  in  the  PMO,  and  any  user  assigned  to  advise 
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the  contractor’s  team.  Input  can  be  provided  on  an  occasional  basis  by  review  teams  and 
focus  groups,  or  on  a  continuing  basis  by  users  in  the  PMO  team. 

User  involvement  includes  the  following: 

•  identifying,  refining  and  validating  requirements 

•  defining  user  capabilities  and  expectations 

•  prioritizing  design  objectives 

•  monitoring,  advising  and  overseeing  design  decisions 

•  deciding  on  design  tradeoffs 

•  evaluating  prototypes  or  models 

•  establishing  validation  and  acceptance  criteria 

There  are  several  means  of  identifying,  and  refining  user  requirements  for  a  new  system. 
These  include  surveys  or  questionnaires,  interviews,  and  experts  (Rouse,  1991).  Focus 
groups  of  experts  or  system  users  can  provide  valuable  insight  into  the  requirements  for  new 
systems  (Jordan,  1994).  Those  requirements  can  be  documented  as  statements  of  desirable 
characteristics,  for  example,  the  need  to  avoid  scrolling  through  many  pages  of  information 
(ibid.).  User  requirements  can  also  be  expressed  as  scenarios  and  mission  analyses  and  as 
system  functions  (see  NATO  AC/243  Panel-8/RSG.14,  1992).  Narrative  mission 
descriptions  and  functional  analyses  proved  to  be  extremely  useful  in  identifying  the  users 
requirements  in  the  ARDS/ADM  project  (Essens,  Beevis  &  Mack,  1994). 

Several  approaches  can  be  taken  to  defining  user  capabilities.  McLaughlin  (1987)  advocates 
‘user  analysis’  (conducted  prior  to  task  analysis),  by  which  a  representative  profile  of  the 
user  group  is  formed  from  results  of  interviews,  from  observations,  and  from  cognitive,  work 
style  and  personality  measures.  User  analysis  for  CCIS  is  a  recent  development,  however, 
and  it  will  be  some  time  before  methods  appropriate  for  non-specialist  designers  are 
established.  What  is  important  is  to  be  aware  that  designers  implicitly  assume  a  user  profile 
when  designing  systems,  and  that  this  profile  may  not  be  appropriate  and  should  be 
formulated  more  explicitly. 

Lacking  a  robust  approach  to  user  profiling,  current  practice  emphasizes  user  input  to 
iterative  development  and  test.  This  approach  can  be  structured  to  reflect  increasing  design 
definition  throughout  the  project  (Engel  &  Townsend,  1989).  Often,  cost  constraints  limit 
the  user  representatives  in  the  PMO  team  to  only  one  or  two  people.  This  may  introduce 
biases  which  must  be  balanced  by  the  opinions  of  other  users  during  concept  and  design 
reviews  (Essens  et  al.,  1994).  Early  activities  could  involve  only  the  expert  users,  who  give 
feedback  for  testing  and  verifying  concepts  and  ideas.  Later,  as  the  design  matures, 
prototyping,  tests,  and  field  trial  activities  should  involve  a  broader  range  of  users  who 
represent  those  who  will  finally  operate  the  system. 


Prototyping  and  Iterative  Development 

A  prototype  is  "the  first  or  primary  type  of  anything;  a  pattern,  model,  standard,  exemplar, 
archetype"  (OED,  1933).  In  software  engineering,  a  prototype  is  often  a  tentative  or 
intermediate  tangible  representation  of  a  (partial)  solution  to  a  problem.  Its  puipose  is  testing 
the  validity  of  the  assumptions  on  which  the  representation  is  based.  Because  it  is  tangible, 
users  are  better  able  to  criticize  the  chosen  solutions.  Prototyping  allows  implementation 
more  rapidly  and  more  safely  in  projects  with  poorly  understood  requirements  or  goals. 
Prototyping  reduces  the  risk  (costs)  of  investing  in  an  ineffective  solution. 
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In  the  human  factors  literature,  prototyping  is  often  equated  with  creating  the  human- 
computer  interface  (Nickerson  &  Pew,  1990).  The  interface  is  where  the  functionality  of  the 
system  becomes  concrete  for  the  user.  In  system  development,  however,  specification  of  the 
interface  is  usually  done  after  the  specification  of  the  requirements.  It  is  a  further  detailing 
out  of  how  the  user-computer  combination  will  do  a  task.  The  prototype  effectively 
communicates  the  'look  and  feel'  of  the  real  system  and  appears  to  the  user  as  if  it  actually 
works.  To  use  prototyping  early  on  in  the  development  of  concepts  requires  a  representation 
that  can  capture  ideas  and  system  concepts  but  stays  away  from  the  details  of  an  interface. 


1  Why  is  prototyping  so  important  to  the  user-centred  development  process? 

Prototyping  is  the  cornerstone  of  a  user-centred  approach  to  software  engineering.  A 
prototype  is  the  means  by  which  users  and  system  designers  reach  a  common  understanding 
of  system  requirements.  Prototyping  encourages  design  iteration  (Beevis  &  St  Denis,  1992). 
Rapid,  evolutionary  prototypes  provide  users  with  the  opportunity  to  set  design  priorities  and 
accept  responsibility  for  the  success  of  their  system.  The  use  of  prototypes  helps  system 
designers  reduce  uncertainty  and  maintain  quality.  Effective  prototyping  speeds  the  system 
development  process  and  builds  user  confidence  in  the  design  team. 


2  When  to  prototype 

Prototyping  should  occur  throughout  the  design  and  testing  of  a  CCIS.  In  the  definition 
stage  of  a  project,  story  boards  may  be  used  to  help  stakeholders  appreciate  what  can  be 
expected  of  an  automated  system.  During  conceptual  design  and  analysis,  prototypes 
establish  and  confirm  requirements3.  Early  prototypes  facilitate  early  testing  and  the  early 
definition  of  measures  of  system  performance.  During  design,  a  prototype  can  validate  a 
previous  analysis  and  serve  to  obtain  user  input.  At  the  end  of  the  design  phase,  a 
deliverable  prototype  can  serve  as  the  design  definition.  In  system  testing,  fully  functional 
prototypes  can  be  examined  under  controlled  conditions  to  confirm  system  effectiveness. 
Every  prototype  should  have  a  clearly  established  purpose  and  set  of  evaluation  criteria; 
these  may  differ  depending  on  the  stage  of  development. 

Prototyping  is  a  well  established  technique  in  custom  system  design.  More  recently,  'off- 
the-shelf  software  is  finding  its  way  into  CCIS.  Combinations  of  commercial  and  custom 
software  provide  less  expensive,  more  quickly  fielded  systems.  In  these  cases,  a  high-level 
prototype  can  be  used  for  software  selection  and  testing  integration  concepts.  Since  hybrid 
systems  provide  less  configuration  control,  the  use  of  prototyping  to  identify  any  limits  on 
meeting  requirements  is  important.  The  description  of  one  recent  DND  CCIS  project 
(Helleur,  Richardson  &  Roy,  1994)  concludes  that  rapid  prototyping  of  a  hybrid  system 
produced  an  initial  system  in  less  than  six  months. 


3  Prototyping  has  its  problems 

Prototyping  is  necessary  for  a  successful  user-centred  approach  to  system  design,  but  it  must 
be  approached  with  caution  and  it  does  not  replace  the  requirements  analysis  process. 


3  McLaughlin  (1987)  has  argued  that  “new  procurement  procedures  are  needed  in  order  to  insure  that  these 
activities  are  conducted  early.  Formal  documentation  deliverables  in  contracts  must  initially  yield  to  the 
delivery  of  prototypes,  and  the  analysis  surrounding  their  development  and  trial  use.”  Such  prototypes  could 
be  accepted  as  ‘samples  and  models’  as  part  of  a  proposal  or  specification,  as  outlined  in  the  CF  instructions  for 
specification  preparation  (DND,  1979). 
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Prototyping  does  not  have  universal  acceptance  among  software  engineers.  Not  all 
applications  of  prototyping  have  been  successful.  There  are  reports  of  a  Canadian  Forces 
project  which  was  "prototyped  to  death."  Some  problems  that  have  been  identified  are: 

•  prototyping  requires  significant  user  involvement,  which  has  to  be  organized  and 
forms  another  management  factor 

•  there  is  no  database  of  prototype  successes,  making  comparisons  difficult 

•  prototyping  is  not  a  design  short  cut,  but  requires  FTERATIONS,  which  may  slow 
the  early  stages  of  system  analysis  and  design 

•  prototyping  evokes  wedding  to  initial  ideas 

•  prototyping  is  less  suitable  for  very  large  projects,  process-oriented  projects  and 
when  test  time  is  limited 

•  prototyping  fails  where  users  insist  on  doing  the  work  the  same  way  as  before 

•  prototyping  fails  when  customized  to  support  several  different  users'  views 

•  prototyping  is  ineffective  when  sub-projects  are  poorly  coordinated,  there  are  poor 
development  standards,  or  inexperienced  staff. 

•  prototyping  efforts  may  not  be  subject  to  formal  test  or  evaluation  (Beevis  &  St 
Denis,  1992). 

Overmyer  (1990)  has  argued  that  negative  reports  of  using  prototypes,  in  particular 
concerning  the  time  spent  on  them,  are  typically  due  to  poor  selection  of  the  methodologies 
and  tools. 


4  Analysis  and  prototyping 

Prototyping  has  a  trial  and  error  characteristic.  To  be  efficient  and  to  avoid  too  many 
obvious  errors  and  backtracking,  the  problem  has  first  to  be  analyzed.  Lack  of  sufficient 
analysis  can  inhibit  the  development  of  a  successful  design  (Melkus  &  Torres,  1988). 
Experience  with  the  ARDS/ADM  project  has  confirmed  the  value  of  analyses  of  system 
missions  (using  narrative  mission  descriptions),  and  system  functions,  as  a  means  of  defining 
operator  functions  and  tasks  (Essens  et  al.,  1994).  Previous  studies  have  shown  that  task 
analysis  can  be  used  effectively  in  the  development  of  a  prototype  (Beevis  &  St  Denis, 

1992). 

Lack  of  sufficient  analysis  can  also  hinder  the  evaluation  of  a  prototype.  This  occurred  in 
the  DCIEM  development  of  the  concept  of  the  SHINMACS  standard  operating  console  for 
ship’s  machinery.  The  concept  was  developed  based  on  a  top-level  analysis  of  operator 
functions  and  tasks  (Gorrell,  1979),  and  developed  using  a  personal  computer  to  prototype 
the  system  screen  designs  and  interactive  protocols.  The  design  of  the  prototype  was  used  to 
establish  the  specification  for  the  Advanced  Development  Model  (ADM)  of  the  console 
(Gorrell,  1980,  1981).  However,  the  operator  functions  and  tasks  were  not  analysed  further, 
so  the  performance  requirements  for  the  ADM  were  not  established  and  could  not  be  used 
for  the  ADM  evaluation. 

However,  although  systems  development  texts  emphasize  the  importance  of  analysis,  an 
analysis  does  not  always  provide: 

•  any  answer 

•  a  useful  answer 

•  a  correct  answer! 

A  prototype  can  help  identify  where  analysis  has  fallen  short  by  forcing  designers  to  make 
decisions  that  may  not  have  been  given  their  due  regard  in  the  analysis  activities. 

Thus,  the  right  balance  must  be  achieved  between  seeking  more  information  about  the 
problem  and  putting  effort  into  representing  what  is  known  with  a  chance  of  failure.  This  is 
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not  always  easy,  as  stated  by  Seaton  and  Stewart,  (1992):  "Putting  it  another  way,  analysis- 
driven  designs  have  tended  to  be  unusable,  whereas  user-driven  designs  have  been 
unbuildable."  In  a  procedure  that  can  be  characterized  as  ‘build  a  little,  test  a  little’,  small 
steps  are  taken  to  simultaneously  find  out  whether  the  solution  addresses  the  goals,  and 
develop  a  further  specification  of  the  goals. 

5  Conclusions  about  prototyping 

In  summary,  prototyping  is  key  to  the  success  of  a  user-centred  approach  to  system  design. 
Prototypes  provide  users  the  means  to  take  responsibility  for  their  system  and  give  designers 
a  tool  for  reducing  uncertainty  and  ensuring  a  quality  design.  To  be  successful,  prototyping 
should  be  rapid  and  evolutionary  in  nature  and  occur  in  all  phases  of  a  project;  unfortunately 
this  is  not  always  the  case.  Prototyping,  when  poorly  applied,  will  fail  and  will  delay  a 
project.  Analysis  and  prototyping  are  integral  and  complementary  techniques  in  system 
design.  A  successful  user-centred  approach  will  use  the  strategy,  ‘build  a  little,  test  a  little.’ 


Testing  Against  Usability  Criteria 

Usability  in  an  information  system  is  a  multi-dimensional  concept  that  includes  elements 
such  as  (ANSI/AIAA,  1992): 

•  level  of  detail  of  information 

•  reliability  of  the  system 

•  accuracy  of  the  information 

•  ability  to  alert  the  user  to  change 

•  built-in  diagnostic  procedures  and  graceful  degradation 

•  non-intrusiveness  into  the  task 

•  operator  acceptance  of  the  system 

•  simplicity  of  information 

•  timeliness  of  information 

•  quality  (i.e.  subjectivity)  of  information 

•  flexibility  of  the  system. 

The  evaluation  of  usability  is  not  simple,  and  includes  both  objective  measures  of 
performance  and  subjective  measures  of  users’  response  to  the  system  (Chapanis,  1991b). 
Evaluation  of  a  command  and  control  system  includes  a  wide  range  of  objective  and 
subjective  measures  (Engel  &  Townsend,  1991;  HumanSystems,  1993;  Sweet,  Metersky  & 
Sovereign,  1986). 

Successful  information  systems  must  include  trials  by  a  sample  of  the  user  community.  In 
CCIS,  field  trials  of  the  system  are  critical  because  field  conditions  are  often  difficult  to 
simulate  for  prototype  testing;  field  trials  may  provide  the  only  opportunity  for  access  to 
portions  of  the  user  community. 

In  a  user-centred  approach  field  trials  occur  under  the  direction  of  a  test  plan  which  includes 
elements  noted  above.  Relative  levels  of  detail  in  the  testing  are  prioritized  and  determined 
by  stakeholders,  including  users.  Results  of  testing  can  be  compared  to  pre-determined 
acceptance  criteria,  or  used  to  establish  system  operating  constraints  4.  Field  tests  lead  to 
specifications  for  future  improvement  and  final  modifications  to  the  design. 


4  The  distinction  between  acceptance  criteria  and  establishing  system  operating  constraints  is  determined  by 
whether  there  is  an  existing  system  and  associated  performance  data.  Separate  field  trials  may  be  necessary 
early  in  system  design  to  determine  performance  data  for  an  existing  system. 
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User-Centred  Approach  and  COTS  Acquisitions 

Much  of  the  foregoing  description  of  a  user-centred  developmental  approach  also  applies  to 
projects  where  COTS  software  and  equipment  are  to  be  used.  The  unqualified  adoption  of 
COTS  would  assume  that  CF  users  have  the  same  capabilities,  training,  and  experience  as 
those  for  which  the  COTS  system  was  designed  and  that  they  use  the  same  doctrine  and 
procedures.  Clearly  this  is  seldom  the  case.  The  basic  HSI  questions  must  still  be  answered 
to  ensure  that  the  system  matches  the  users.  This  means  that  the  users  and  their  capabilities 
and  skills  must  be  clearly  defined  to  bidders  and  that  the  evaluation  of  any  proposed  COTS 
must  take  a  user-centred  approach.  As  noted  in  the  description  of  the  system  development 
process,  the  system  specification  must  include  HSI  and  HFE  requirements. 

The  US  Government  has  developed  a  set  of  questions  to  determine  the  extent  to  which 
human  factors  were  considered  in  the  system  acquisition  process  (US  GAO,  1981).  Those 
questions  emphasize  personnel  and  training  but  also  include  HFE;  they  might  be  used  to 
audit  HSI  issues  for  COTS  acquisitions.  For  example,  auditors  are  required  to  determine 
whether,  during  the  full  scale  engineering  development  phase  of  a  project,  “man-machine 
tradeoff  criteria  were  used  in  the  development  of  operation  and  maintenance  ... 
control/display/software  design  concepts  ...  [and  to]  ...  assess  the  extent  to  which 
developmental  tests  demonstrated  (1)  system  conformance  with  human  factors  engineering 
design  criteria,  (2)  the  adequacy  of  the  training  approach,  (3)  acceptability  of  system- 
imposed  operator  workload,  and  (4)  system  safety.” 

The  US  Army  has  developed  a  much  more  detailed  approach  to  HSI  for  non-developmental 
item  acquisition  (US  ARI,  1988).  Although  directed  at  the  US  Army’s  MANPRINT 
programme,  the  overall  process  may  provide  the  basis  for  COTS  acquisitions  for  DND.  The 
approach  has  five  phases  (Marton  &  Toomer,  1989): 

Phase  1  -  Identification  of  mission  requirements  and  constraints 

Phase  2  -  Identification  and  description  of  required  item  characteristics  and  selection 
criteria 

Phase  3  -  Search  or  survey  to  identify  potential  candidates  and  collect  information 
related  to  required  characteristics 

Phase  4  -  Assessment  and  comparison  of  candidates  against  criteria,  and  selection  of 
system 

Phase  5  -  Test  and  development  (in  exceptional  cases)  to  improve  operational, 
survivability,  cost,  scheduling  or  other  features 
For  conventional  systems,  identification  and  description  of  the  HFE  features  can  be  achieved 
through  the  completion  by  bidders  of  the  Human  Engineering  Design  Approach  Document  - 
Operator  (HEDAD-O)  and  the  Human  Engineering  Design  Approach  Document  - 
Maintainer  (HEDAD-M).  Those  deliverables  are  specified  by  US  DoD  DIDs,  respectively 
DI-HF AC-80746  and  DI-HFACE-80747  (US  DoD,  1987).  The  DIDs  require  the  bidder  to 
describe  the  layout,  detail  design  and  arrangement  of  crew  station  equipment  and  operator 
and  maintainer  tasks.  Although  comprehensive  HEDADs  are  extremely  useful  for  describing 
systems  for  HFE  evaluation,  the  dynamic  aspects  of  CCIS  require  evaluations  to  include 
prototypes,  dynamic  mockups,  or  actual  system  operation.  A  draft  DID  covering  rapid 
prototyping  activities  has  been  prepared,  but  has  not  yet  been  adopted  as  a  standard  (Steiner 
&Ims,  1991). 

Projects  which  use  COTS  do  not  necessarily  acquire  a  complete  system.  It  may  be  that  there 
are  opportunities  to  take  advantage  of  available  COTS  components  such  as  a  geographical 
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information  system,  a  relational  data  base,  or  a  commercial  word  processor.  In  those  cases 
developmental  effort  is  still  required  and  the  overall  approach  to  development  outlined  in 
this  report  is  relevant. 


DRAFT  GENERIC  HEPP 


Introduction 

The  basis  of  any  planning  for  HFE  activities  is  the  project  Statement  of  Requirement  (SOR) 
and  Request  for  Proposals  (RFP).  If  the  PMO  does  not  make  clear  the  importance  of  user 
requirements  in  the  SOR,  and  of  HFE  in  the  RFP,  then  the  contractor  cannot  be  expected  to 
place  any  importance  on  those  aspects  in  their  work  (NATO  AC/243  Panel-8, 1984). 
Whether  the  systems  have  been  designed  already  or  have  yet  to  be  developed,  the  HFE 
activities  should  be  the  subject  of  a  project  plan,  usually  referred  to  as  the  Human 
Engineering  Programme  Plan  (HEPP). 

The  HEPP  defines  the  requirements  for  applying  human  engineering  to  a  project,  including 
the  work  to  be  accomplished  in  conducting  a  human  engineering  effort  integrated  with  the 
total  system  engineering  and  development  effort.  The  HEPP  provides  the  basis  for  including 
HFE  during  proposal  preparation,  system  analysis,  task  analysis,  system  design  (including 
computer  software  design),  equipment  and  facilities  design,  testing,  and  documentation  and 
reporting. 

Experience  in  the  application  of  HFE  in  a  number  of  DND  projects  suggests  that  the  scope 
of  the  HFE  activities  is  strongly  related  to  the  preparation  of  a  HEPP  (Beevis,  1987). 
Experience  with  the  ARDS/AJDM  project  supports  the  importance  of  the  HEPP  (Essens  et  al. 
,  1994).  The  HEPP  should  cover  the  following  (Beevis  et  al.,  1993): 

•  work  items:  stakeholder  identification 
requirements  development 

analysis  -  scenarios,  missions,  functions,  operator  tasks 

design  and  development 

interface  specification 

prototyping 

iterative  development 

test  and  evaluation 


•  relationship  of  HFE  activities  to  other  systems  development  activities  (SSS  review, 
etc.) 

•  schedule 

•  meetings 

•  reviews 

•  deliverables:  plans,  progress  reports  and  results  of  studies,  tests,  trials  and 
evaluations. 

The  plan  must  be  compatible  with  the  documents  shown  in  Table  2. 
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Table  2:  Documents  relevant  to  HFE  In  CCIS 

REGULATORY 

•  DND  PM  Handbook,  CFP  A-LP-005-000/AG,  chapter  35 

•  US  M1L-H-46855B  Human  Engineering  Requirements  for  Military  Systems,  Equipment  and 
Facilities 

•  US  MIL-STD-490  -  Specification  Practices 

•  US  MIL-STD-881 A  -  Work  Breakdown  Structures  for  Defense  Material  Items 

•  US  MIL-STD  1388-2B  -  Logistic  Support  Analysis 

•  US  MIL-STD- 152 IB  (USAF)  -  Technical  reviews  &  audits  for  systems,  equipments  and 
computer  software 

•  US  MIL-STD  2167A  -  Defense  System  Software  Development 

•  METHOD/1™ 

GUIDANCE 

•  NATO  AC/243(PaneI-8)TR/7  -  Analysis  techniques  for  man-machine  systems  design, 
Volumes  1  &  2 

•  NATO  AC/243( Panel-8)  RSG.19  report  -  Toward  a  framework  for  Cognitive  Analysis, 
Design,  and  Evaluation 

•  UK  Human  factors  guidelines  for  the  design  of  computer-based  systems,  Volumes  1  -  6, 
HMSO 

•  Human  factors  in  command-and-control  system  procurements,  (Copas,  Triggs,  &  Manton, 
1985) 

•  Guidelines  for  the  design  and  evaluation  of  operator  interfaces  for  computer  based  control 
systems,  (Engel  &  Townsend,  1989) 


Work  Items 

In  DND  projects,  the  HEPP  is  usually  based  on  the  DID  DI-HFAC-80740  ‘Human 
engineering  program  plan’  (US  DoD,  1987).  The  tasks  to  be  included  in  the  plan  are 
specified  by  US  MEL-H-46855B  ‘Human  engineering  requirements  for  military  systems, 
equipment  and  facilities’  (US  DoD,  1979).  That  specification  “establishes  and  defines  the 
requirements  for  applying  human  engineering  to  the  development  and  acquisition  of  military 
systems,  equipment  and  facilities”  including  “the  work  to  be  accomplished  by  the  contractor 
or  subcontractor  in  conducting  a  human  engineering  effort  integrated  with  the  total  system 
engineering  and  development  effort”  as  “the  basis  for  including  human  engineering  during 
proposal  preparation,  system  analysis,  task  analysis,  system  design  (including  computer 
software  design),  equipment  and  facilities  design,  testing,  and  documentation  and  reporting.” 

The  work  items  in  MEL-H-46855B  are  compatible  with  the  usual,  sequential,  systems 
development  process  ('Table  3).  The  work  items  start  with  the  analysis  of  functions  and 
concentrate  on  the  traditional  areas  of  human  engineering.  As  Overmyer  (1990)  has  noted, 
MIL-H-46855B  does  not  preclude  an  iterative  approach  to  system  development,  and 
provides  a  starting  point  for  the  development  of  a  generic  human  factors  engineering  plan  for 
CCIS. 
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Table  3  :  Work  Items  in  US  MIL-H-46855B 

1.  ANALYSIS 

•  Defining  and  allocating  system  functions 

•  Information  flow  and  processing  analysis 

•  Estimates  of  potential  operator/maintainer  processing  capabilities 

•  Allocation  of  functions 

•  Equipment  selection 

•  Analysis  of  tasks 

•  Preliminary  system  and  subsystem  design 

2.  DESIGN  AND  DEVELOPMENT 

•  Human  engineering  in  equipment  detail  design 

•  Studies,  experiments  and  laboratory  tests 

•  Equipment  detail  design  drawings 

•  Work  environment,  crew  stations  and  facilities  design 

•  Human  engineering  in  performance  and  design  specifications 

•  Equipment  procedure  development 

3.  TEST  AND  EVALUATION 

•  Planning 

•  Implementation 

•  Failure  analysis 


Comparison  of  Table  1  with  Table  3  shows  that  the  major  differences  between  the  user- 
centred  approach  advocated  for  CCIS  and  MIL-H-46855B  is  in  the  emphasis  placed  on 
requirements  development  and  initial  analyses.  Some  of  this  work  must  be  undertaken  by  the 
procuring  agency,  as  discussed  below.  With  those  requirements  in  mind,  an  initial  approach 
to  a  generic  HEPP  for  CCIS  has  been  developed  from  US-MIL-H-46855B,  with  necessary 
amplifications  and  explanation,  as  set  out  below. 
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HEPP  based  on  US  MIL-H-46855B 
MEL-H-46855B  REQUIREMENT 


1.1  Scope  -  This  specification  establishes  and  defines  the 
requirements  for  applying  human  engineering  to  the 
development  and  acquisition  of  military  systems,  equipment 
and  facilities.  These  requirements  include  the  work  to  be 
accomplished  by  the  contractor  or  subcontractor  in  conducting 
a  human  engineering  effort  integrated  with  the  total  system 
engineering  and  development  effort.  These  requirements  are 
the  basis  for  including  human  engineering  during  proposal 
preparation,  system  analysis,  task  analysis,  system  design 
(including  computer  software  design),  equipment  and  facilities 
design,  testing,  and  documentation  and  reporting. 


1.2  Applicability  -  It  is  not  intended  that  all  the  requirements 
contained  herein  should  be  applied  to  every  program  or 
program  phase.  In  accordance  with  DoD  principles,  directives 
and  regulations  governing  the  application  and  tailoring  of 
specifications  and  standards  to  achieve  cost  effective 
acquisition  and  life  cycle  ownership  of  defense  materiel,  this 
specification  shall  be  tailored  to  specific  programs  and  the 
milestone  phase  of  the  program  within  the  overall  life  cycle. 
This  tailoring  shall  be  the  selected  application  of  methods, 
tables,  sections,  individual  paragraphs  or  sentences,  or  a 
combination  thereof,  to  be  placed  on  contract  in  order  to 
impose  only  the  minimum  essential  needs  to  preclude 
unnecessary  and  unreasonable  program  costs.  Guidance  for 
selection  by  the  procuring  activity  of  this  specification  for 
contract  use,  and,  when  invoked,  the  partial  and  incremental 
application  of  the  requirements  provisions,  is  contained  in  the 
Appendix. 

2.  APPLICABLE  DOCUMENTS 


COMMENTS,  ADDITIONS 
OR  CHANGES  REQUIRED 

The  work  items  needed  for  a 
user-centred  approach  cover  a 
wider  scope.  More  emphasis 
must  be  placed  on  requirements 
definition,  including  die 
development  of  scenarios  and 
mission  analyses.  Some  of  this 
may  be  the  responsibility  of  the 
procuring  agency  rather  than  the 
contractor.  Thus,  the  overall 
HEPP  for  a  CCIS  project  should 
include  procuring-agency 
activities,  and  the  PMO  and 
contractor  should  use  the  HEPP 
to  define  their  relative 
responsibilities. 

In  view  of  the  need  for  iteration 
in  the  development  of  CCIS, 
there  must  be  a  clear  agreement 
on  the  number  of  cycles  of 
iteration  to  be  undertaken  “to 
impose  only  the  minimum 
essential  needs  to  preclude 
unnecessaiy  and  unreasonable 
program  costs”,  see  Morgan 
1989. 


Other  documents  could  be 

referenced,  for  example: 

•  US  Defense  Information 
Systems  Agency  Human 
Computer  Interface  Style 
Guide,  Version  2.0,  September 
30,  1992 

•  Guidelines  for  Designing  User 
Interface  Software 

•  NATO  AC/243  (Panel- 
8)TR/7,  (1992).  Analysis 
Techniques  for  Man-machine 
System  Design. 
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3.  REQUIREMENTS 
3.1  General  Requirements 


The  scope  and  nature  of  the  work 
should  include  the  identification 
of  user  requirements  for  the 
projected  system.  This  may 
require  the  addition  of  a  sub¬ 
section  on  Requirements 
Definition. 


3.1.1  Scope  and  Nature  of  Work  -  Human  engineering  shall  be 
applied  during  development  and  acquisition  of  military 
systems,  equipment  and  facilities  to  achieve  the  effective 
integration  of  personnel  into  the  design  of  the  system.  A 
human  engineering  effort  shall  be  provided  to  develop  or 
improve  the  crew-equipment/software  interface  and  to  achieve 
required  effectiveness  of  human  performance  during  system 
operation/  maintenance/control  and  to  make  economical 
demands  upon  personnel  resources,  skills,  training  and  costs. 
The  human  engineering  effort  shall  include,  but  not  necessarily 
be  limited  to,  active  participation  in  the  following  three  major 
interrelated  areas  of  system  development. 


Activities  necessary  for 
requirements  analysis  and 
definition  must  be  added.  These 
activities  should  be  interrelated 
with  the  analysis,  design,  and 
test  and  evaluation  activities  to 
support  iterative  development. 
Test  and  evaluation  should 
include  story-boarding  and 
prototyping  efforts. 


3.1.2  Human  Engineering  Program  Planning  -  Human 
Engineering  Program  Planning,  in  accordance  with  the 
requirements  of  this  specification  and  the  equipment 
specification,  shall  include  the  tasks  to  be  performed,  human 
engineering  milestones,  level  of  effort,  methods  to  be  used, 
design  concepts  to  be  utilized,  and  the  test  and  evaluation 
program,  in  terms  of  an  integrated  effort  within  the  total 
project. 


The  activities  of  requirements 
definition,  concept  development, 
allocation  of  functions,  and 
concept  exploration  must  be 
integrated  with  Design,  Test  & 
Evaluation  activities.  The  extent 
of  iteration  anticipated  in  the 
HFE  activities  should  be 
identified.  Such  planning  should 
include  the  preparation  of  a  User 
Participation  Plan,  a  Rapid 
Prototyping  Plan,  and  a  Plan  for 
Resolving  Issues. 


3.1.3  Nonduplication  -  The  efforts  performed  to  fulfill  the  No  changes  required. 

human  engineering  requirements  specified  herein  shall  be 

coordinated  with,  but  not  duplicate  efforts  performed  in 

accordance  with  other  contractual  requirements.  Necessary 

extensions  or  transformations  of  the  results  of  other  efforts  for 

use  in  the  human  engineering  program  will  not  be  considered 

duplication.  Instances  of  duplication  or  conflict  shall  be 

brought  to  the  attention  of  the  Contracting  Officer. 
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3.2.1  Analysis  -  Mission  analysis  shall  be  developed  from  a 
baseline  scenario.  Analysis  shall  include  application  of  human 
engineering  techniques  as  follows: 


3.2.1. 1  Defining  and  Allocating  System  Functions  -  The 
functions  that  must  be  performed  by  the  system  in  achieving 
its  objective(s)  within  specified  mission  environments  shall  be 
analyzed.  Human  engineering  principles  and  criteria  shall  be 
applied  to  specify  personnel-equipment/  software  performance 
requirements  for  system  operation,  maintenance  and  control 
functions  and  to  allocate  system  functions  to  (1)  automatic 
operation/maintenance,  (2)  manual  operation/maintenance,  or 
(3)  some  combination  thereof.  Function  allocation  is  an 
iterative  process  achieving  the  level  of  detail  appropriate  for 
the  level  of  system  definition. 

3.2.1. 1.1  Information  Flow  and  Processing  Analysis  - 
Analyses  shall  be  performed  to  determine  basic  information 
flow  and  processing  required  to  accomplish  the  system 
objective  and  include  decisions  and  operations  without 
reference  to  any  specific  machine  implementation  or  level  of 
human  involvement. 


This  section  should  refer  to  the 
need  for  iterative  Requirements 
Development,  and  should  cover: 

•  User  Analysis 

•  System,  organizational  & 
training  analysis 

•  Specification  of  performance 
goals  &  criteria. 

The  ARDS/ADM  project  has 
confirmed  the  value  of  scenarios 
(strictly,  narrative  mission 
descriptions)  prepared  and 
reviewed  from  the  users’ 
perspective  (Essens  et  al.,  1994). 
The  preparation  of  scenarios  and 
mission  analyses  is  best  done  by 
experienced  CF  personnel.  Thus 
responsibility  for  scenario 
preparation  must  be  made  clear. 
System  scenarios  could  be 
deliverables. 

This  section  covers  the  analysis 
of  system  functions.  The  section 
can  be  used  to  cover  the  “ 
specification  of  performance 
goals  &  criteria”  as  required  in 
Table  3.  The  extent  of  iteration 
needed  to  achieve  “the  level  of 
detail  appropriate  for  the  level  of 
system  definition”  must  be 
agreed  upon  and  defined. 


Data  Flow  Diagrams,  State 
Transition  Diagrams,  etc.  which 
are  often  used  in  the  analysis  and 
description  of  proposed 
information  systems  can  be  used 
for  this  activity.  There  is  an 
opportunity  here  to  show  how 
the  human  engineering  analysis 
effort  will  be  integrated  with 
other  systems  engineering  efforts 
to  avoid  duplication  of  effort. 


P144780.PDF  [Page:  33  of  60] 


-23  - 


3.2.1. 1.2  Estimates  of  Potential  Qperator/Maintainer 
Processing  Capabilities  -  Plausible  human  roles  (e.g.,  operator, 
maintainer,  programmer,  decision  maker,  communicator, 
monitor)  in  the  system  shall  be  identified.  Estimates  of 
processing  capability  in  terms  of  load,  accuracy,  rate  and  time 
delay  shall  be  prepared  for  each  potential  operator/maintainer 
information  processing  function.  These  estimates  shall  be  used 
initially  in  determining  allocation  of  functions  and  shall  later 
be  refined  at  appropriate  times  for  use  in  definition  of 
operator/maintainer  information  requirements  and  control, 
display  and  communication  requirements.  In  addition, 
estimates  shall  be  made  of  the  effects  on  these  capabilities 
likely  to  result  from  implementation  or  nonimplementation  of 
human  engineering  design  recommendations.  Results  from 
studies  in  accordance  with  3.2.2. 1  may  be  used  as  supportive 
inputs  for  these  estimates. 

3.2. 1.1. 3  Allocation  of  Functions  -  From  projected 
operator/maintainer  performance  data,  estimated  cost  data,  and 
known  constraints,  the  contractor  shall  conduct  analyses  and 
tradeoff  studies  to  determine  which  system  functions  should  be 
machine-implemented  or  software  controlled  and  which 
should  be  reserved  for  the  human  operator/maintainer. 


In  practice  it  is  difficult  to  make 
such  estimates  of  processing 
capability,  and  few  analyses 
include  the  type  of  information 
required  (Beevis  ,  1987).  The 
section  should  be  re-written  to 
cover  the  User  Analysis  which  is 
a  feature  of  a  user-centred 
approach.  This  would  identify 
the  potential  user’s  capabilities 
in  ways  related  to  the  task,  for 
example,  whether  the  users  have 
typing  skills. 


Computer-based  information 
systems  can  either  mechanize  the 
transfer  of  information,  without 
changing  operator  functions,  or 
they  can  change  the  allocation  of 
functions  by  automating  operator 
tasks  associated  with  sensing, 
collating  information,  or  decision 
making.  This  is  the  issue  of 
‘automating  a  manual  system  vs. 
building  an  automated  system’ 
(Beevis  et  al.,  1993).  To  deal 
with  the  allocation  of  functions, 
the  ARDS/ADM  project 
developed  a  two-stage  function 
allocation  process,  wherein 
functions  are  allocated  initially 
to  humans  or  to  software,  and 
those  which  are  ambiguous  are 
re-examined  for  implementation 
by  a  human  assisted  by  software 
(Essens  et  al.,  1994). 
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3. 2. 1.2  Equipment  Selection  -  Human  engineering  principles 
and  criteria  shall  be  applied  along  with  all  other  design 
requirements  to  identify  and  select  the  particular  equipment  to 
be  operated/maintained/  controlled  by  personnel.  The  selected 
design  configuration  shall  reflect  human  engineering  inputs, 
expressed  in  quantified  or  "best  estimate"  quantified  terms,  to 
satisfy  the  functional  and  technical  design  requirements  and  to 
insure  that  the  equipment  will  meet  the  applicable  criteria 
contained  in  MIL-STD- 1472,  as  well  as  other  human 
engineering  criteria  specified  by  the  contract. 


MIL-STD- 1472D  does  not  deal 
thoroughly  with  all  aspects  of 
human-computer  interaction. 
Another  standard  should  be 
identified  and  used  in  the  SOR, 
or  a  user  interface  style  guide 
should  be  agreed  upon.  The 
style  should  be  based  on  user 
profiles.  In  ARDS/ADM  it  was 
agreed  that  the  TCCCS 
communications  back  plane 
would  be  used,  which  determines 
some  of  the  user  interface. 
However,  the  Took  and  feel’  of 
an  interface  is  a  function  of  the 
dynamics  of  operator  use,  and 
emerges  from  the  interaction 
between  user  and  machine. 
Therefore  it  cannot  be 
completely  determined  by 
written  standards,  and  must  be 
verified  by  prototyping. 


3-2. 1.3  Analysi£...Q.f  Tasks  -  Human  engineering  principles  and  This  sub-section  requires  no 

criteria  shall  be  applied  to  analyses  of  tasks.  modification. 


3.2.1. 3.1  Gross  Analysis  of  Tasks  -  The  analyses  shall  provide 
one  of  the  bases  for  making  design  decisions:  e.g., 
determining,  to  the  extent  practicable,  before  hardware 
fabrication,  whether  system  performance  requirements  can  be 
met  by  combinations  of  anticipated  equipment,  software,  and 
personnel,  and  assuring  that  human  performance  requirements 
do  not  exceed  human  capabilities.  These  analyses  shall  also  be 
used  as  basic  information  for  developing  preliminary  manning 
levels;  equipment  procedures;  skill,  training  and 
communication  requirements;  and  as  Logistic  Support 
Analysis  inputs,  as  applicable.  Those  gross  tasks  identified 
during  human  engineering  analysis  which  are  related  to  end 
items  of  equipment  to  be  operated  or  maintained  by  personnel 
and  which  require  critical  (see  6.2.1)  human  performance, 
reflect  possible  unsafe  practices  or  are  subject  to  promising 
improvements  in  operating  efficiency  shall  be  further 
analyzed,  with  the  approval  of  the  procuring  activity. 


The  gross  analysis  of  tasks 
should  be  used  to  identify  those 
tasks  which  require  prototyping, 
because  they  “require  critical 
(see  6.2.1)  human  performance, 
reflect  possible  unsafe  practices 
or  are  subject  to  promising 
improvements  in  operating 
efficiency.” 
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3.2. 1.3.2  Analysis  of  Critical  Tasks  -  Further  analysis  of 
critical  tasks  shall  identify  the:  (1)  information  required  by 
operator/maintainer,  including  cues  for  task  initiation;  (2) 
information  available  to  operator/  maintainer;  (3)  evaluation 
process;  (4)  decision  reached  after  evaluation;  (5)  action  taken; 
(6)  body  movements  required  by  action  taken;  (7)  workspace 
envelope  required  by  action  taken;  (8)  workspace  /available; 
(9)  location  and  condition  of  the  work  environment;  (10) 
frequency  and  tolerances  of  action;  (11)  time  base;  (12) 
feedback  informing  operator/  maintainer  of  the  adequacy  of 
actions  taken;  (13)  tools  and  equipment  required;  (14)  number 
of  personnel  required,  their  specialty  and  experience;  (15)  job 
aids  or  references  required;  (16)  communications  required, 
including  type  of  communication;  (17)  special  hazards 
involved;  (18)  operator  interaction  where  more  than  one  crew 
member  is  involved;  (19)  operational  limits  of  personnel 
(performance);  and  (20)  operational  limits  of  machine  and 
software.  The  analysis  shall  be  performed  for  all  affected 
missions  and  phases  including  degraded  modes  of  operation. 

3.2. 1.3.3  Workload  Analysis  -  Individual  and  crew  workload 
analysis  shall  be  performed  and  compared  with  performance 
criteria. 


3.2. 1.3.4  Concurrence  and  Availability  -  Analyses  of  tasks 
shall  be  modified  as  required  to  remain  current  with  the  design 
effort  and  shall  be  available  to  the  procuring  activity. 


3.2. 1.4  Preliminary  System  and  Subsystem  Design  -  Human 
engineering  principles  and  criteria  shall  be  applied  to  system 
and  subsystem  designs  represented  by  design  criteria 
documents,  performance  specifications,  drawings  and  data, 
such  as  functional  flow  diagrams,  system  and  subsystem 
schematic  block  diagrams,  interface  control  drawings,  overall 
layout  drawings  and  related  applicable  drawings  provided  in 
compliance  with  contract  data  requirements.  The  preliminary 
system  and  subsystem  configuration  and  arrangement  shall 
satisfy  personnel-equipment/software  performance 
requirements  and  comply  with  applicable  criteria  specified  in 
MIL-STD-1472  as  well  as  other  human  engineering  criteria 
specified  by  the  contract. 


Contractors  seldom  provide  all 
the  information  called  up  in  this 
section  (Beevis,  1987).  Critical 
tasks  should  be  examined  by 
prototyping,  or  simulation, 
wherever  possible. 


The  section  could  be  modified  to 
include  the  comparison  of  results 
from  prototyping  or  simulation 
with  predictions  of  operator 
workload  and  performance 
criteria.  In  addition,  the  section 
should  address  the  issue  of 
system  manning,  and  the 
preparation  of  the  system 
manning  plan. 

No  such  clause  governs  the  Test 
and  Evaluation  efforts.  The 
clause  should  be  modified  to 
state  that  analyses  of  functions, 
tasks,  and  interface  design 
requirements  shall  be  modified 
to  remain  current  with  the  design 
and  prototyping  effort. 

This  section  should  include 
reference  to  rapid  prototypes. 
The  requirement  “The 
preliminary  system  and 
subsystem  configuration  and 
arrangement  shall  satisfy 
personnel-equipment/software 
performance  requirements  and 
comply  with  applicable  criteria 
specified  in  MIL-STD-1472D  as 
well  as  other  human  engineering 
criteria  specified  by  the  contract” 
should  specifically  mention  the 
target  users  and  the  training  plan. 
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3.2.2  Human  Engineering  in  Equipment  Detail  Design  - 
During  detail  design  of  equipment,  the  human  engineering 
inputs,  made  in  complying  with  the  analysis  requirements  of 
paragraph  3.2.1  herein,  as  well  as  other  appropriate  human 
engineering  inputs,  shall  be  converted  into  detail  equipment 
design  features.  Design  of  the  equipment  shall  meet  the 
applicable  criteria  of  MIL-STD-1472  and  other  human 
engineering  criteria  specified  by  the  contract.  Human 
engineering  provisions  in  the  equipment  shall  be  evaluated  for 
adequacy  during  design  reviews.  Personnel  assigned  human 
engineering  responsibilities  by  the  contractor  shall  participate 
in  design  reviews  and  engineering  change  proposal  reviews  of 
equipment  end  items  involving  personnel  interfaces.  Human 
engineering  requirements  during  equipment  detail  design  are 
specified  in  paragraphs  3.2.2.1,  3.2.2.2,  3.2.2.3,  3.2.2.4  and 
3.2.2.5  herein. 

3.2.2. 1  Studies,  Experiments  and  Laboratory  Tests  -  The 
contractor  shall  conduct  experiments,  tests  (including  dynamic 
simulation  per  paragraph  3.2.2.1.2),  and  studies  required  to 
resolve  human  engineering  and  life  support  problems  specific 
to  the  system.  Human  engineering  and  life  support  problem 
areas  shall  be  brought  to  the  attention  of  the  procuring  activity, 
and  shall  include  the  estimated  effect  on  the  system  if  the 
problem  is  not  studied  and  resolved.  These  experiments,  tests, 
and  studies  shall  be  accomplished  in  a  timely  manner,  i.e., 
such  that  the  results  may  be  incorporated  in  equipment  design. 
The  performance  of  any  major  study  effort  shall  require 
approval  by  the  procuring  activity. 

3.2.2. 1.1  Mockups  and  Models  -  At  the  earliest  practical  point 
in  the  development  program  and  well  before  fabrication  of 
system  prototypes,  full-scale  three-dimensional  mockups  of 
equipment  involving  critical  human  performance  shall  be 
constructed.  The  proposed  Human  Engineering  Program  Plan 
shall  specify  mockups  requiring  procuring  activity  approval 
and  modification  to  reflect  changes.  The  workmanship  shall  be 
no  more  elaborate  than  is  essential  to  determine  the  adequacy 
of  size,  shape,  arrangement,  and  panel  content  of  the 
equipment  for  human  use.  The  most  inexpensive  materials 
practical  shall  be  used  for  fabrication.  These  mockups  and 
models  shall  provide  a  basis  for  resolving  access,  workspace 
and  related  human  engineering  problems,  and  incorporating 
these  solutions  into  systems  design.  Upon  approval  by  the 
procuring  activity,  scale  models  may  be  substituted  for 
mockups.  In  those  design  areas  where  systems/equipment 
involve  critical  human  performance  and  where  human 
performance  measurements  are  necessary,  functional  mockups 
shall  be  provided,  subject  to  prior  approval  by  the  procuring 
activity.  The  mockups  shall  be  available  for  inspection  as 
determined  by  the  procuring  activity.  Disposition  of  mockups 
and  models,  after  they  have  served  the  purposes  of  the 
contract,  shall  be  as  directed  by  the  procuring  activity. 


The  requirement  could  be 
expanded  to  cover  prototyping  of 
the  interfaces.  Any  of  the 
following  three  sub-sections 
could  address  prototyping. 
Although  “Mockups  and 
Models”  and  “Dynamic 
Simulation”  could  cover 
prototyping  activities,  they  are  a 
test  or  experimental  activity. 


This  sub-section  should  be 
expanded  to  include  any  planned 
prototyping  activities.  The 
outcome  of  prototyping  can  be 
described  by  the  relevant  DID 
(DI-HF AC-80743  Human 
Engineering  Test  Plan). 


This  requirement  need  not  be 
modified  if  prototyping  is  dealt 
with  under  3.2.2. 1.  Mockups  and 
models  may  have  some  value  in 
testing  and  training.  Reference 
should  be  made  to  such  uses. 
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3 .2.2. 1.2  Dynamic  Simulation  -  Dynamic  simulation 
techniques  shall  be  utilized  as  a  human  engineering  design  tool 
when  necessary  for  the  detail  design  of  equipment  requiring 
critical  human  performance.  Consideration  shall  be  given  to 
use  of  various  models  for  the  human  operator,  as  well  as  man- 
in-the-loop  simulation.  While  the  simulation  equipment  is 
intended  for  use  as  a  design  tool,  its  potential  relationship  to, 
or  use  as,  training  equipment  shall  be  considered  in  any  plan 
for  dynamic  simulation. 

3 -2.2.3  Work  Environment.  Crew  Stations  and  Facilities 
Assign  -  Human  engineering  principles  and  criteria  shall  be 
applied  to  detail  design  of  work  environments,  crew  stations 
and  facilities  to  be  used  by  system  personnel.  Drawings, 
specifications  and  other  documentation  of  woric  environment, 
crew  stations  and  facilities  shall  reflect  incorporation  of  human 
engineering  requirements  and  compliance  with  applicable 
criteria  of  MTL-STD- 1472  and  other  human  engineering 
criteria  specified  by  the  contract.  Design  of  work  environment, 
crew  stations  and  facilities  which  affect  human  performance, 
under  normal,  unusual  and  emergency  conditions,  shall 
consider  at  least  the  following  where  applicable: 

a.  Atmospheric  conditions,  such  as  composition,  volume, 
pressure  and  control  for  decompression,  temperature,  humidity 
and  air  flow. 

b.  Weather  and  climate  aspects,  such  as  hail  snow,  mud,  arctic, 
desert  and  tropic  conditions. 

c.  Range  of  accelerative  forces,  positive  and  negative, 
including  linear,  angular  and  radial. 

d.  Acoustic  noise  (steady  state  and  impulse),  vibration,  and 
impact  forces. 

e.  Provision  for  human  performance  during  weightlessness. 

f.  Provision  for  minimizing  disorientation. 

g.  Adequate  space  for  personnel,  their  movement,  and  their 
equipment. 

h.  Adequate  physical,  visual,  and  auditory  links  between 
personnel  and  personnel  and  their  equipment,  including  eye 
position  in  relation  to  display  surfaces,  control  and  external 
visual  areas. 

i.  Safe  and  efficient  walkways,  stairways,  platforms  and 
inclines. 

j.  Provisions  for  minimizing  psychophysiological  stresses. 

k.  Provisions  to  minimize  physical  or  emotional  fatigue,  or 
fatigue  due  to  work-rest  cycles. 


It  seems  best  to  leave  this  sub¬ 
section  to  cover  activities  such 
as  computer  simulation  of 
operator  task  networks  (i.e., 
analysis)  and  not  to  include 
prototyping. 


The  requirement  emphasizes 
physical  aspects  of  the  design, 
and  not  all  the  items  covered  are 
relevant  to  CCIS. 
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l.  Effects  of  clothing  and  personal  equipment,  such  as  full  and 
partial  pressure  suits,  fuel  handler  suits,  body  armor,  polar 
clothing,  and  temperature  regulated  clothing. 

m.  Equipment  handling  provisions,  including  remote  handling 
provisions  and  tools  when  materiel  and  environment  require 
them. 


n.  Protection  from  chemical,  biological,  toxicological, 
radiological,  electrical  and  electromagnetic  hazards. 

o.  Optimum  illumination  commensurate  with  anticipated 
visual  tasks. 


p.  Sustenance  and  storage  requirements  (i.e.,  oxygen,  water 
and  food),  and  provision  for  refuse  management; 

q.  Crew  safety  protective  restraints  (shoulder,  lap  and  leg 
restraint  systems,  inertia  reels  and  similar  items)  in  relation  to 
mission  phase  and  control  and  display  utilization. 


3.2.2;4  Human  Engineering  in  Performance  and  Design 
Specifications  -  The  provisions  of  performance  and  design 
specifications  prepared  by  the  contractor,  shall  conform  to 
applicable  human  engineering  criteria  of  MIL-STD-1472  and 
other  human  engineering  criteria  specified  by  the  contract. 


MIL-STD-1472D  establishes 
design  standards,  not 
performance  criteria.  Overall 
human  factors  performance 
criteria  for  the  system  should  be 
established  during  the 
requirements  definition  phase. 
The  text  should  be  modified  to 
reflect  this. 


3.2.2.5  Equipment  Procedure  Development  -  Based  upon  the  No  changes  required. 

human  performance  functions  and  tasks  identified  by  human 

engineering  analyses  (3.2.1  herein),  the  contractor  shall  apply 

human  engineering  principles  and  criteria  to  the  development 

of  procedures  for  operating,  maintaining  or  otherwise  using  the 

system  equipment.  For  computer  systems  where  operating  and 

maintenance  procedures  are  largely  determined  by  software 

programs,  human  engineering  shall  be  applied  throughout  the 

software  program  planning  and  development.  This  effort  shall 

be  accomplished  to  assure  that  the  human  functions  and  tasks 

identified  through  human  engineering  analysis  are  organized 

pd  sequenced  for  efficiency,  safety  and  reliability,  to  provide 

inputs  to  the  Logistic  Support  Analysis  where  required,  and  to 

assure  that  the  results  of  this  effort  shall  be  reflected  in  the 

development  of  operational,  training  and  technical 

publications. 
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3-2.3  Human  Engineering  in  Test  and  Evaluation  -  The 
contractor  shall  establish  and  conduct  a  test  and  evaluation 
program  to:  (1)  assure  fulfillment  of  the  applicable 
requirements  herein;  (2)  demonstrate  conformance  of  system, 
equipment  and  facility  design  to  human  engineering  design 
criteria;  (3)  confirm  compliance  with  performance 
requirements  where  personnel  are  a  performance  determinant; 

(4)  secure  quantitative  measures  of  system  performance  which 
are  a  function  of  the  human  interaction  with  equipment;  and 

(5)  determine  whether  undesirable  design  or  procedural 
features  have  been  introduced.  (The  fact  that  these  functions 
may  occur  at  various  stages  in  system,  subsystem,  or 
equipment  development  shall  not  preclude  final  human 
engineering  verification  of  the  complete  system.  Both  operator 
and  maintenance  tasks  shall  be  performed  as  described  in 
approved  test  plans  during  the  final  system  test.) 


3.2.3. 1  Planning  -  Human  engineering  testing  shall  be 
incorporated  into  the  system  test  and  evaluation  program  and 
shall  be  integrated  into  engineering  design  and  development 
tests,  contractor  demonstrations,  flight  tests,  R&D  acceptance 
tests  and  other  development  tests.  Compliance  with  human 
engineering  requirements  shall  be  tested  as  early  as  possible. 
Human  engineering  findings  from  design  reviews,  mock-up 
inspections,  demonstrations  and  other  early  engineering  tests 
shall  be  used  in  planning  and  conducting  later  tests.  Human 
engineering  test  planning  shall  be  directed  toward  verifying 
that  the  system  can  be  operated,  maintained,  supported  and 
controlled  by  user  personnel  in  its  intended  operational 
environment  Test  planning  shall  include  methods  of  testing 
(e.g.,  use  of  checklists,  data  sheets,  test  participant  descriptors, 
questionnaires,  operating  procedures  and  test  procedure) , 
schedules,  quantitative  measures,  test  criteria  and  reporting 
processes. 


Test  and  Evaluation  (T&E)  has 
the  following  aims: 

1.  Assure  fulfillment  of  the 
applicable  requirements 

2.  Demonstrate  conformance  of 
the  system,  equipment,  and 
facility  design  to  human 
engineering  design  criteria, 

3.  Confirm  compliance  with 
performance  requirements 
where  personnel  are  a  major 
determinant 

4.  Secure  quantitative  measures 
of  system  performance ..... 

5.  Determine  whether 
undesirable  features  have  been 
introduced. 

The  requirement  should  be 
revised  to  include: 

•  Concept  demonstration 

•  Assurance  of  operability  by  the 
intended  users  in  the  intended 
environment 

•  Demonstration  of  the 
effectiveness  of  the  Training 
Plan 

•  That  the  planned  activities  will 
be  integrated  with  those  of 
requirements  definition  and 
prototyping. 

The  main  thrust  to  Test  and 
Evaluation  (T&E)  is  to 
demonstrate  that  the  system  will 
meet  performance  requirements. 
Given  the  need  for  an  iterative 
approach  to  the  development  of 
CCIS,  T&E  activities  should 
occur  throughout  the 
development  cycle.  The  T&E 
plan  should  reflect  this,  and 
show  how  performance 
requirements  will  be  identified 
and  tested.  The  plan  must  be 
cross-referenced  to  the  user 
participation  activities,  plans  for 
rapid  prototyping,  and  the 
Training  Plan. 


P144780.PDF  [Page:  40  of  60] 


-30  - 


3 .2.3.2  Implementation  -  The  human  engineering  test  and  The  first  item  could  be  expanded 
evaluation  plan,  shall  be  implemented  upon  approval  by  the  to  include  prototyping, 
procuring  activity.  Test  documentation  (e.g.,  checklists,  data 
sheets,  test  participant  descriptors,  questionnaires,  operating 
procedures,  test  procedures)  shall  be  available  at  the  test  site. 

Human  engineering  portions  of  all  tests  shall  include  the 
following: 

a.  A  simulation  (or  actual  conduct  where  possible)  of  mission 
or  work  cycle). 

b.  Tests  in  which  human  participation  is  critical  as  defined  in 
paragraph  6.2.1. 

c.  A  representative  sample  of  non-critical  scheduled  and 
unscheduled  maintenance  tasks  that  do  not  duplicate  the  tasks 
selected  for  the  maintainability  demonstration. 

d.  Proposed  job  aids,  new  equipment  training  (NET)  programs, 
training  equipment,  and  special  support  equipment. 

e.  Utilization  of  personnel  who  are  representative  of  the  range 
of  the  intended  military  user  populations  in  terms  of  skills,  size 
and  strength  and  wearing  suitable  military  garments  and 
equipment  which  are  appropriate  to  the  tasks  and  approved  by 
the  procuring  activity.  (Use  of  military  personnel  from  the 
intended  user  population  is  preferred  where  feasible.) 

f.  Collection  of  task  performance  data  in  simulated  or,  where 
possible,  actual  operational  environments. 

g.  Identification  of  discrepancies  between  required  and 
obtained  task  performance. 


h.  Criteria  for  acceptable  performance  of  the  test. 


3.2.3. 3  Failure  Analysis  -  All  failures  occurring  during  test  and 
evaluation  shall  be  subjected  to  a  human  engineering  review  to 
differentiate  between  failures  due  to  equipment  alone, 
personnel-equipment  incompatibilities  and  those  due  to  human 
error.  The  contractor  shall  notify  the  procuring  activity  of 
design  conditions  which  may  contribute  substantially  to  human 
error  and  shall  propose  appropriate  solutions  to  these 
conditions. 


Failures  should  include  software- 
related  failures  which  might  be 
due  to  human  factors  such  as 
poor  interface  design  or  short 
term  memory  failure  in  handling 
alphanumerics. 
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3.2.4  .Cognizance  and  Coordination  -  The  human  engineering  No  changes  required. 

program  shall  be  coordinated  with  maintainability,  system 

safety,  reliability,  survivability/  vulnerability,  facilities 

engineering,  integrated  logistic  support,  and  other  human 

factors  engineering  functions  including  bio-medical,  life 

support,  personnel  and  training,  and  shall  be  integrated  into  the 

total  system  program.  Results  of  human  engineering  test  and 

evaluation  shall  be  incorporated  into  the  Logistic  Support 

Analysis  Record  (LSAR)  as  applicable.  The  human 

engineering  portion  of  any  analysis,  design  or  test  and 

evaluation  program  shall  be  conducted  under  the  direct 

cognizance  of  personnel  assigned  human  engineering 

responsibility  by  the  contractor. 

3.3  Data  Requirements  -  All  human  engineering  data  No  changes  required, 

requirements  shall  be  as  specified  by  the  contract. 

3.3.1  Traceability. -  The  contractor  shall  appropriately  No  changes  required 

document  his  human  engineering  efforts  to  provide  traceability 

from  the  initial  identification  of  human  engineering 
requirements  during  analysis  and/or  system  engineering 
through  design  and  development  to  the  verification  of  these 
requirements  during  test  and  evaluation  of  approved  design, 
software  and  procedures. 

3.3.2  Access.  -  All  data,  such  as  plans,  analyses,  design  review  No  changes  required 
results,  drawings,  checklists,  design  and  test  notes,  and  other 

supporting  back  ground  documents  reflecting  human 
engineering  actions  and  decision  rationale,  shall  be  maintained 
and  made  available  at  the  contractor's  facilities  to  the 
procuring  activity  for  meetings,  reviews,  audits, 
demonstrations,  test  and  evaluation,  and  related  functions. 

3.4  Drawing  Approval  -  Personnel  assigned  human 
engineering  responsibility  by  the  contractor  shall  approve  all 
layouts  and  drawings  having  potential  impact  on  human 
interface  with  the  system,  equipment,  or  facility. 


4.  QUALITY  ASSURANCE 

Compliance  with  the  requirements  of  this  specification  and 
other  human  engineering  requirements  specified  by  the 
contract  will  ultimately  be  demonstrated  by  the  system's  ability 
to  meet  its  mission  and  operational  objectives.  During  the 
development  program,  compliance  with  the  human 
engineering  requirements,  as  they  pertain  to  system  design  and 
effectiveness,  will  be  demonstrated  at  the  scheduled  design 
and  configuration  reviews  and  inspections  as  well  as  during 
development  test  and  evaluation  inspections,  demonstrations 
and  tests. 


Should  include  all 
representations  of  the  user 
interface,  including  story  boards, 
prototypes,  mockups,  and 
simulations. 

No  changes  required 
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6.1  Intended  Use  -  This  specification  may  be  invoked  in  its 
entirety  or  selectively  as  prescribed  by  the  procuring  activity. 
The  primary  use  of  this  specification  for  procurement  does  not 
necessarily  preclude  its  utilization  for  in-house  efforts,  where 
desired.  Compliance  with  this  specification  will  provide  the 
procuring  activity  with  assurance  of  positive  management 
control  of  the  human  engineering  effort  required  in  the 
development  and  acquisition  of  military  systems,  equipment 
and  facilities.  Specifically,  it  is  intended  to  assure  that: 


No  changes  required 


a.  System  requirements  are  achieved  by  appropriate  use  of  the 
human  component. 


b.  Through  proper  design  of  equipment,  software  and 
environment,  the  personnel-equipment/software  combination 
meets  system  performance  goals. 

c.  Design  features  will  not  constitute  a  hazard  to  personnel. 

d.  Trade-off  points  between  automated  vs.  manual  operation 
have  been  chosen  for  peak  system  efficiency  within 
appropriate  cost  limits. 


e.  Human  engineering  applications  are  technically  adequate. 

f.  The  equipment  is  designed  to  facilitate  required 
maintenance. 


g.  Procedures  for  operating  and  maintaining  equipment  are 
efficient,  reliable  and  safe.  . 

h.  Potential  error-inducing  equipment  design  features  are 
minimized. 

i.  tiie  layout  of  the  facility  and  the  arrangement  of  equipment 
affords  efficient  communication  and  use. 

j.  The  contractors  provide  the  necessary  manpower  and 
technical  capability  to  accomplish  the  above  objectives. 


Relationship  to  other  systems  development  activities 

As  noted  earlier,  a  complete  user-centred  approach  to  CCIS  acquisition  or  development 
includes  work  items  such  as  stakeholder  identification  and  scenario  development  which  mav 
be  performed  best  by  the  PMO.  The  complete  HEPP  for  a  project  would  include  those 
PMOtieS>  an<1  ^  contractor  s  HEPP  must  make  it  clear  what  inputs  they  expect  from  the 


th,e,?ain,stream  systems  engineering  activities  (NATO  AC/243 
(.  ei-5)  IK/7,  1992)  and  should  be  also  associated  with  logistics  support  and  personnel 

sub-system  development  However,  those  activities  must  be  integrated  if  full  benefit  is  to  be 
obuuned  from  them  The  HEPP,  the  systems  engineering  managgnent  plan,  aid  *e soC?e 
development  plan  should  be  delivered  as  part  of  the  contractor’s  proposal  package. 


SeJe^ri?  refers  to  the  TraininS  Plan  and  to  a  T&E  plan.  The  list  of  deliverables 
O  able  6)  also  refers  to  a  ‘manning  and  training  plan’  and  a  ‘training  plan.’  The  Training 
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and  T&E  Plans  may  be  separate  documents,  as  they  are  dealt  with  separately  from  HFE  hv 

STO-Br8r2B‘0MIL^TO-1521B)Ch  by  flK  PM°  ^  MIL-STO^90a!miL- 


Schedule,  Meetings,  Reviews  and  Deliverables 


The  review  points  and  deliverables  in  the  HFE  process  are  listed  in  Table  4  The 
acceptance  plan  and  the  plans  for  user  participation  and  rapid  prototiing,  wMch 
mentioned  m  the  generic  HEPP,  should  be  an  integral  part  of  the  HEPP  They  are 
deliverables  which  should  be  specified  in  the  HEPP.  ^ 


Table  4:  Review  points  and  deliverables  in  the  User  Engineering  Process 

(from  Beevis  et  al.,  1993) 

1.  CONTRACT  READINESS  REVIEW  (CRR) 


a.  Issues  to  be  examined 

•  Human  Engineering  Program  Plan 

•  Stakeholder  &  User  Participation 

•  Organization  of  User  Engineering  Steering 


2.  SYSTEM  REQUIREMENTS  REVIEW 
(SRR)  (Preliminary  user-design  review) 

issues  to  be  examined 

•  Workflows 

•  Organization 

•  System  Concept 

•  Allocation  of  Functions 


3.  SYSTEM  DESIGN  REVIEW  (SDR) 
(Includes  User  Design  Review) 

Issues  to  be  examined 
■  Work  station  details 


4.  PRELIMINARY  DESIGN  REVIEW  (PDR) 


Information  to  be  reported 

•  Scenarios  &  performance  specifications 

•  Mission  and  Function  Analyses 

•  Prototypes 

•  Draft  manning  &  training  plan 

•  Plan  for  resolving  issues 

•  Draft  requirements  specification 


Information  to  be  reported 

•  Prototype  baseline  and  performance  data 

•  Requirements  specification 

•  Training  plan 

•  User  acceptance  plan 


Information  to  be  reported 

5.  CRITICAL  DESIGN  REVIEW  (PDR)  ’  Interface  standards  &  guidelines 

Information  to  be  reported 
•  Design  configuration  documents 

6.  SYSTEM  ACCEPTANCE 

•  Data  obtained  from  replaying 
scenarios  from  SRR 
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Evaluation  of  proposed  HEPP 

HEPP  are  best  evaluated  with  the  assistance  of  subject  matter  experts  representing  both 
human  factors  specialists  and  project  specialists.  When  evaluating  a  contractor’s  HEPP 
project  personnel  must  consider  a  number  of  factors: 

Technical  proposal: 

•  Does  the  proposal  show  understanding  of  scope  and  objectives  of  the  work? 

•  Is  the  proposed  approach  sound? 

•  Does  the  plan  identify  the  input  required  from  PMO  and  system  users? 

•  Does  the  HEPP  show  the  relationship  between  HFE,  manpower,  personnel  and 
training,  system  safety,  test  and  evaluation,  and  systems  engineering  activities"'* 

•  Work  items: 

-  do  they  propose,  or  have  they  completed,  HFE  work  items,  as  identified  in 
Table  1? 

-  how  will  user  input  be  obtained? 

-  what  is  the  completion  action  for  each  work  item?  does  it  include  a  report? 

-  how  will  acceptance  criteria  for  the  work  item  products  be  defined? 

-  do  they  propose  a  list  of  deliverables  as  outlined  in  Table  4? 

•  Is  the  timescale  and  level  of  effort  practical? 

•  Are  personnel  loadings  shown  for  each  activity? 

•  Are  regular  review  points  scheduled? 

•  Will  the  results  of  the  HFE  effort  be  timely  for  the  overall  project  effort? 

Team  capabilities  &  experience: 

•  Does  the  team  include  a  human  factors  specialist? 

•  What  experience  does  the  team  have  in  user-centred  development? 

•  What  experience  do  they  have  in  designing  for  military  users? 

•  Have  they  experience  in  performance  measurement  and  user  trials? 


CONCLUSIONS  AND  RECOMMENDATIONS 

Human  Factors  Engineering  has  gained  wide  acceptance  in  the  design  of  the  human 
computer  interface.  However,  most  software  development,  whether  custom  or  COTS,  has  yet 
to  fully  integrate  HFE  techniques  into  the  software  engineering  process.  A  case  can  be 
developed  for  the  inclusion  of  HFE  in  CCIS  development  on  the  basis  of  either  usability  or 
cost  This  report  was  written  to  address  the  integration  of  HFE  with  the  development  of 
CCIS,  to  present  the  user-centred  development  process,  and  to  propose  a  draft  of  a  generic 
project  plan  for  the  application  of  HFE  in  CCIS. 

Much  of  the  documentation  governing  the  development  of  systems  and/or  software  does  not 
mmaS1Ze  or  Observation  of  the  ARDS/ADM  project  showed  that  even  with 
PMO  encouragement,  the  integration  of  HFE  with  other  systems  engineering  activities  is  far 
from  assured. 

A  review  of  several  approaches  to  the  development  of  system  software  showed  that  they  do 
not  place  a  strong  emphasis  on  HFE  or  user  requirements.  New  approaches  to  software 

could  facilitate  the  integration  of  HFE  in  the  system  engineering  process.  The 
METHOD/1  toolset,  being  adopted  by  DND,  combined  with  a  move  toward  more  object- 
oriented  analysis,  design,  and  coding,  offer  attractive  opportunities  to  integrate  HFE  in 
software  engineering.  It  is  concluded,  that  HFE  can  be  included  into  various  approaches  to 
software  development  provided  that  such  activities  are  planned  for. 
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User-centred  design  is  a  new  development  in  software  engineering  which  emphasizes  the 
human  in  system  design.  While  several  different  approaches  to  user-centred  design  have 
been  reported  recendy,  all  emphasize  user  involvement  in  the  design  process,  prototyping 
and  iterative  development,  and  testing  against  usability  criteria.  Scenarios,  mission  analyses 
and  functional  decompositions  are  very  effective  for  obtaining  user  input.  Prototyping  is  the 
key  to  user-centred  design,  but  it  must  be  applied  with  skill  and  towards  specific  goals,  and  it 
must  be  balanced  by  analysis. 

An  effective  Human  Engineering  Programme  Plan  documents  the  agreement  between 
contractor  and  procuring  agency  on  the  extent  and  scope  of  HFE  activities  and  their 
relationship  to  other  systems  engineering  efforts.  US  MIL-H-46855B  contains  the  essential 
ingredients  of  a  good  HEPP,  but  some  additions  are  required.  The  HEPP  should  be  detailed, 
but  information  requirements  should  be  realistic  and  unnecessary  studies  avoided.  The  HEPP 
should  define  human  engineering  activities  to  be  carried  out  during  project  phases  and  it 
should  specify  deliverables.  Manning,  training,  and  testing  issues  should  be  considered. 

Other  standards  and  references  can  be  invoked  by  the  HEPP  for  specific  issues. 

There  is  a  need  to  continue  to  emphasize  the  integration  of  HFE  in  CCIS.  CRAD/DRDL  is 
sponsoring  work  at  DCIEM  aimed  at  the  development  of  a  course  in  HFE  for  project 
managers.  The  work  will  also  revise  and  expand  the  draft  HEPP  included  in  this  report.  If 
the  METHOD/1  software  design  methodology  becomes  required  by  DND  for  CF  acquisition 
projects,  then  the  creation  of  HFE  segments,  tasks  and  work  steps  should  be  investigated. 
Observations  of  the  ARDS/ADM  project  will  continue  under  contract  to  DCIEM  and  the 
results  will  be  used  to  further  develop  this  plan  to  integrate  HFE  in  CCIS,  in  a  process  that  is 
intended  to  be  acceptable  to  project  managers  and  contractors. 
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LIST  OF  ABBREVIATIONS 

ADM  -  Associate  Deputy  Minister 
ARDS  -  Artillery  Regimental  Data  System 
CASE  -  Computer  Aided  Software  Engineering 
CCIS  -  Command  and  Control  Information  System 
COTS  -  Commercial  Off-the-Shelf 
CRAD  -  Chief  of  Research  and  Development 
CSCI  -  Computer  Software  Configuration  Item 
CSC  -  Computer  Software  Component 
CSU  -  Computer  Software  Unit 

DCIEM  -  Defence  and  Civil  Institute  of  Environmental  Medicine 

DID  -  Data  Item  Description 

DND  -  Department  of  National  Defence 

DoD  -  US  Department  of  Defense 

DPM  -  Deputy  Project  Manager 

DPMS  -  Defence  Programme  Management  System 

DRDL  -  Director,  Research  and  Development,  Land 

HE  -  Human  Engineering 

HEMP  -  Human  Engineering  Management  Plan 

HEPP  -  Human  Engineering  Project  Plan 

HFE  -  Human  Factors  Engineering,  synonymous  with  Human  Engineering 

HMS  O  -  Her  Majesty  ’  s  S  tationary  Office 

H-P  -  Hatley-Pirbhai 

HSI  -  Human  Systems  Integration 

HWCI  -  Hardware  Configuration  Item 

IAW  -  In  Accordance  With 

MANPRINT  -  Manpower  and  Personnel  Integration 

OOP  -  Object-Oriented  Programming 

PM  -  Project  Manager 

PMO  -  Project  Management  Office 

QA  —  Quality  Assurance 

RA  -  Requirements  Analysis 

RFP  -  Request  for  Proposals 

SDM  -  System  Development  Methodology 

SDR  -  System  Design  Review 
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SE  -  System  Engineering 

SOR  -  Statement  of  Requirement 

SOW  -  Statement  of  Work 

SRA  -  System  Requirements  Analysis 

SRR  -  System  Requirements  Review 

SSS  -  System  Software  Specification 

TAD  -  Target  Audience  Description 

TCCCS  -  Tactical  Command,  Control  and  Communications  System 

T&E  -  Test  and  Evaluation 

TRLO  -  Technical  Requirements  Liaison  Officer 
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GLOSSARY  OF  TERMS  AND  DEFINITIONS 

allocation  of  functions  —  See  function  allocation. 

analysis  —  The  resolution  of  anything  complex  into  its  simple  elements. 

contractor  -  An  organization,  usually  in  industry,  which  contracts  to  perform  engineering 
activities  to  develop  and  build  a  system  or  equipment. 

critical  task  -  A  task  which,  if  not  accomplished  in  accordance  with  system 
requirements,  will  have  adverse  effects  on  cost,  system  reliability,  efficiency 
effectiveness,  or  safety  (after  US  MIL-H-46855B). 


data  item  description  (DID)  -  a  form  used  to  describe  the  data  required  from  a 
contractor,  each  DID  being  listed  in  the  Contract  Data  Requirements  List  (CDRL). 

design  and  development  -  The  phase  of  an  equipment  programme  which  calls  for  design 
engineering  work  aimed  at  full  validation  of  the  technical  approach  and  ensures  complete 
system  integration  to  the  point  where  production  contract  action  can  be  taken  (NATO 


designer  -  One  who  designs  or  plans  or  makes  patterns  for  manufacture. 

equipment  -  All  non-expendable  items  needed  to  outfit/equip  an  individual  or 
organization  (NATO  Glossary).  '  J 

ergonomics  —  The  systematic  study  of  the  relation  between  the  human,  machine,  tools, 
and  environment,  and  the  application  of  anatomical,  physiological,  and  psychological 
knowledge  to  the  problems  arising  therefrom.  Synonymous  with  Human  Factors. 

function  —  A  broad  category  of  activity  performed  by  a  system,  usually  expressed  as  a 
verb  +  noun  phrase,  e.g.,  “control  air- vehicle,”  “update  way-point”  (NATO  STANAG 
3994/1).  A  function  is  a  logical  unit  of  behaviour  of  a  system. 

function  allocation  -  The  process  of  deciding  how  system  functions  shall  be 
implemented  -  by  human,  by  equipment,  or  by  both  -  and  assigning  them  accordingly. 

functional  analysis  —  An  analysis  of  system  functions  describing  broad  activities  which 
may  be  implemented  by  personnel,  and/or  hardware  and/or  software. 

human  engineering  (HE)  -  The  area  of  human  factors  which  applies  scientific 
TTc°iwrn  u  design  of  items  to  achieve  effective  human-machine  integration  (after 
Ub  MIL-H-46855B).  Human  engineering  includes  developmental  test  and  evaluation 
activities. 

human  factors  (HF)  —  A  body  of  scientific  facts  about  human  capabilities  and 
limitations.  It  includes  principles  and  applications  of  human  engineering,  personnel 
selection,  training,  life  support,  job  performance  aids,  and  human  performance  evaluation 
Synonymous  with  Ergonomics. 


human-machine  system  —  A  composite  of  equipment,  related  facilities,  material, 
software  and  personnel  required  for  an  intended  operational  role. 
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S,1ns“dM  c™i"fonMUal  resP°nsible  fOT  raining  a  defence  system  in,  or  restoring  i, 
manpower  -  The  demand  for  human  resources  in  terms  of  numbers  and  organization. 
“f^rcaMlSgof0'1'  USUaDy  ‘°  ““  “  fo™  set  of  methods 

in  response  to  a 

S ^requMto'ct^/a  operational  capabilities  of  military  forces 

and/or  postulated  threat  with  an  acceptable  deg^f  of  ri“k  (NATO^sT  °f  ‘h6 

rC™Usmctio?rfrLd1n  “sS  a  new  dSCiionm  1 °r  neap°n'  used  in  studying 
(UK  DBF  STAN  00  25)  *  physicaI  characteristics  of  a  system  or  sub-system 

teoperato’S’SdnSS  “?=hara.cte™aos  of  equipment  that  permits 

safety,  under  the  intended  onera  HnnC'  Sonne  of  average  skill,  with  maximum 
performance  (DND-ENG.  STD-3)  ltlons’  to  obtain  the  required  operational 

m“o„:  asniensfgVnedUal  Primalily  reSP°nsible  for  usin*  *  astern,  or  enabling  a  system 
ph(StwSEsdeflniti0n  °f  manpower  in  terms  of  trade,  skill,  experience  levels,  and 


function  for  a  specffied 
NATO 

SSsSSsSS?"-?1 s="“  - 

checked  (AGARD-  Multilingual  Dictionary).  d  Whereby  conformance  can  be 
and  accurately  ‘iaSCTi^^e^ssentiafanirteclmica/requhemem^forltems.^MterialsCor 
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services,  including  procedures  by  which  it  can  be  determined  that  the  requirements  have 
been  met  (CAN  A-LP-005-000/AG-006). 


standard  -  An  exact  value,  a  physical  entity,  or  an  abstract  concept,  established  and 
defined  by  authority,  custom,  or  common  consent  to  serve  as  a  reference,  model,  or  rule  in 
measuring  quantities,  establishing  practices  or  procedures,  or  evaluating  results  A  fixed 

quantity  or  quality  (NATO  Glossary  of  Terms  and  Definitions). 

.  -  A  document  that  establishes  engineering  and  technical  limitations  and 

(CAN  A^P  (WS^OOO/vSi^Ob)8,  processes’  methods>  designs,  and  engineering  practices 


statement  of  requirement  (SOR)  -  A  statement  of  the  capability  required  of  a  new 
systTem,  to  meet  an  existing  or  postulated  threat,  synonymous  with  NATO  Staff  Target  In 
the  UK  it  includes  estimated  costs  and  technical  factors. 

story  board  -  A  sequence  of  displays,  usually  paper  drawings,  which  illustrate  how  a 
system  may  appear  when  operated. 

system  —  In  general  a  set  or  arrangement  of  things  so  related  or  connected  as  to  form  a 
unity  or  organic  whole  (Webster  s  New  World  Dictionary  of  the  American  Language,  2nd 
College  Edition,  1970.  The  Publishing  Company). 

system  design  -  The  preparation  of  an  assembly  of  methods,  procedures,  and  techniques 
united  by  regulated  iterations  to  form  an  an  organized  whole  (NATO  Glossary  of  Terms). 

system  effectiveness  —  The  probability  that  the  system  will  provide,  in  terms  of  resources 
required,  and  as  specified,  either: 

(a)  the  maximum  operational  performance  within  the  total  cost  prescribed 
or 

(b)  the  required  value  at  lowest  cost.  (CAN  DND-ENG-STD-3). 

system(s)  engineering  (SE)  -  A  basic  tool  for  systematically  defining  the  equipment, 
personnel,  facilities  and  procedural  data  required  to  meet  system  objectives  (US  MIL-H- 


system  requirements  analysis  (SRA)  —  An  analysis  of  what  is  required  of  a  system  to 
identify  those  characteristics  which  the  system  (both  personnel  and  equipment)  must  have 
to  satisfy  the  purposes  of  the  system  (after  UK  DEF  STAN  00-25). 


task  -  A  composite  of  related  operator  or  maintainer  activities  (perceptions,  decisions, 
and  responses)  performed  for  an  immediate  purpose,  e.g.,  “insert  aircraft  position”  (after 
NATO  STANAG  3994/1).  F  v 


task  analysis  -  A  time  oriented  description  of  personnel-equipment-software  interactions 
brought  about  by  an  operator,  controller  or  maintainer  in  accomplishing  a  unit  of  work 
with  a  system  or  item  of  equipment.  It  shows  the  sequential  and  simultaneous  manual  and 
intellectual  activities  of  personnel  operating,  maintaining,  or  controlling  equipment  (US 
MIL-H-46855B). 


Test  and  Evaluation  (T&E)  -  A  comprehensive  programme  of  test  activities,  conducted 
throughout  the  system  hierarchy  and  over  the  system  life  cycle,  to: 
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(a)  assess  system  performance 

(b)  verify  conformance  to  system  requirements 

(c)  determine  system  acceptability 

training  -  The  process  by  which  trainees  acquire  or  enhance  specific  sldlk  imrra/krW 
and  attitudes  required  to  accomplish  military  tasks  P  ’ 

usability  -  the  set  of  attributes  that  bear  on  the  effort  required  to  use  fleam  understand 
set  ofPus?“(Aplo,ai99“)i  °n  Mvidual  ^“ssment  of  such  use  by  a  skied  or  implied 

maSSJ SySt6m  ~  A  combinati°n  of  one  or  more  weapons  with  all  related  equipment 

teanf acdWdes^ed  for^t^ean^nagement!^  suk~systems  design/development 

of  “ operator  to  meet  ^ 
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APPENDIX  A:  LIST  OF  DIDs  INCLUDED  IN  US  DOD  STD-2167-A 


System/Segment  Design  Document  (SSDD) 
Software  Development  Plan  (SDP) 

Software  Requirements  Specification  (SRS) 
Interface  Requirements  Specification  (IRS) 
Interface  Design  Document  (IDD) 

Software  Design  Document  (SDD) 

Software  Product  Specification  (SPS) 

Version  Description  Document  (VDD) 
Software  Test  Plan  (STP) 

Software  Test  Description  (STD) 

Software  Test  Report  (STR) 

Computer  System  Operator’s  Manual  (CSOM) 
Software  User’s  Manual  (SUM) 

Software  Programmer’s  Manual  (SPM) 
Firmware  Support  Manual  (FSM) 


DI-CMAN-80534 
DI-CMAN-80030A 
DI-MCCR-80025A 
DI-MCCR-80026A 
DI-MCCR-80027A 
DI-MCCR-800 12A 
DI-MCCR-80029A 
DI-MCCR-800 1 3 A 
DI-MCCR-800 14A 
DI-MCCR-80015A 
DI-MCCR-800 17 A 
DI-MCCR-800 1 8 A 
DI-MCCR-800 1 9 A 
DI-MCCR- 8002 1 A 
DI-MCCR-80022A 


Computer  Resources  Integrated  Support  Document  (CRISD)  DI-MCCR-80024A 
Engineering  Change  Proposal  (ECP)  DI-E-3128 

Specification  Change  Notice  (SCN)  DI-E-3134 
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